Tuesday, November 1, 2011

Results of the 2011 ClojureCLR Survey

The 2011 ClojureCLR Survey concluded last night.  Inspired by Chas Emerick's State of Clojure Surveys (2010, 2011), the survey was an attempt to get feedback from those interested in ClojureCLR regarding usage, problems, needs and desires, in the hope of setting the direction for ClojureCLR development.

The survey responses are summarized below.  I will provide my analysis in a later posting.

The survey was announced by three posts to the Clojure mailing list and two tweets from my Twitter account.  Since I had never tweeted before (I may be anti-social), I made sure a few select people were following so that there would be minimum of retweeting.

There were 53 responses to the survey.  This number is 8% of the 670 responses to the 2011 State of Clojure survey.  For comparison, the ClojureCLR repo has 29% the number of watchers of the Clojure repo.  There is no way to know how representative this sample in the universe of Clojure folk.

Make of this what you will:  Given ten days to respond,  36% of the responses came on the last day of the survey.  :)

The first two questions on the survey were an attempt to judge the respondent's past and present engagement with ClojureCLR.


What is your level of experience with ClojureCLR?




27 (51%) -- Never met
21 (40%) -- Dated a few times
 2 ( 4%) -- Hanging out
 2 ( 4%) -- Entertaining thoughts about a future together
 1 ( 2%) -- Raising a family
The majority of respondents have never used ClojureCLR.  Just 9% have used it more than a few times.


How would you characterize your use of ClojureCLR today?
 


27 (51%) -- Really, I've never touched it.
16 (30%) -- I tried it, gave it up.
 8 (15%) -- I'm just tinkering.
 2 ( 4%) -- I use it for serious "hobby" projects.
 0 ( 0%) -- I use it at work. 


"Never met" was almost perfectly correlated with "Really, I've never touched it", as it should be. "Dated a few times" was unfortunately mostly paired with "I tried it, gave it up". (Some of the other pairings suggest I should have been less cute with the labels.)

Never
touched
Tried it
gave it up
Tinkering Serious
"hobby"
Work
Never met
25
1
1
Dated a few times
1
15
5
Hanging out
1
1
Thoughts of a
future together
1
1
Raising a family
1

At any rate, it is clear that the level of involvement is very low overall.


Which of the following apply to your Microsoft development experience?



14 (27%) -- I presently develop on the .Net platform for non-work projects.
27 (52%) -- I presently develop on the .Net platform at work.
26 (50%) -- I have tried other non-Microsoft languages targeting the .Net platform.
15 (29%) -- My workplace does not use the .Net platform.
16 (31%) -- My workplace allows non-Microsoft languages targeting the .Net platform.
 8 (15%) -- My workplace permits ClojureCLR for development.
 8 (15%) -- My workplace would never permit ClojureCLR for development. 



What has been most frustrating for you in your use of ClojureCLR or kept you from using it (more or at all)? (Over and above general Clojure issues.)

This question asked the respondent to rank each of a set of categories on a 1-5 scale of importance.

Not  imp.
1

2

3

4
Very imp.
5
Availability of ClojureCLR-specifc documentation
5
8
11
8
13
ClojureCLR's install process
8
4
12
7
12
Difficulty deploying completed applications
11
8
12
5
4
Clojure/contrib libs not ported
5
4
11
11
11
Lack of integration with Visual Studio
14
6
9
7
7
Availability of ClojureCLR-targeted editors/dev environments
10
8
9
6
10
Startup speed
13
8
7
4
6
Runtime performance
9
6
10
9
5
Interop support
5
8
8
9
11
Concern about lack of community
3
3
14
11
13
Concern about quality
10
7
6
6
14
Concern about long-term viability
6
3
5
13
16
Mono
13
5
6
6
11

Focusing on the  total number of responses of 4+5 or 3+4+5 is one way to get a ranking of relative importance.

3+4+5Rank4+5Rank
Concern about lack of community
38
1

24
2
Concern about long-term viability
34
2

29
1
Clojure/contrib libs not ported
33
3

22
3
Availability of ClojureCLR-specifc documentation
32
4

21
4
ClojureCLR's install process
31
5

19
7
Interop support
28
6

20
5 (T)
Concern about quality
26
7

20
5 (T)
Availability of ClojureCLR-targeted editors/dev environments
25
8

16
9
Runtime performance
24
9

14
10 (T)
Mono
23
10 (T)

17
8
Lack of integration with Visual Studio
23
10 (T)

14
10 (T)
Difficulty deploying completed applications
21
12

9
13
Startup speed
17
13

10
12

The top four in each ranking contain the same items.  It is possible that people who tried ClojureCLR early and were disappointed in the install process and interop support would be happier with these in the most recent version;  this deserves investigation.  Because I suspected that the questions about community and viability would be high on the list, later questions probed these topics more.  See below.

This question allowed additional comments.  Among the deterents noted:
  • In general the lack of examples of typical uses, something which is pretty well covered in the JVM based Clo[j]ure, things like jetty/ring, jdbc, and GUI (which tends to be a lot more important in the windows world).
  • Emacs support as good as standard clojure's is extremely important! Support for .net generics is extremely important! Lack of these two features were the show stoppers for me. 
  • Lack of ported libraries is also pretty important.  
  • I haven't even really seen anything in terms of guidance / best practices to allow this work to be carried forward by others in a consistent manner.
  • Also, it would be great if there were more readily available progress reports, roadmaps, etc from David et al.  Until this survey became available, ClojureCLR hasn't had a blog, so that's a good start.
  • VsClojure is unreliable. Trying to build the self generated main() works maybe one or two times, then keeps failing when the bootstrap compiler aborts. I guess this might be covered by the vs integration question but it turns off developers immediately. 
  • Microsoft can kill off support at any time. Linux is a 2nd class citizen. .NET/CLR is not a viable target for anything.
  • The completeness of the ClojureCLR implementation compared to the Clojure implementation. I.e. do they do the same thing? Is there a Clojure Spec suite of tests or something like that that both implementations pass?
  • Lack of personal need for CLR.
  • Generally the install seems bulky compared to clojure for the jvm.
  • I hadn't heard of it. I'm quite interested in Clojure, but was not aware of a CLR port. So greatest impediment is lack of knowledge/lack of PR.
  •  Mono is my insurance policy, so I want at a minimum as much .NET code as I write (to a maximum of ""all .NET code I write and use"") to run on Mono as well on MS's CLR. [...] I'd love to have ClojureCLR work seamlessly on Mono.
Rate the following for desirability.
Respondents were asked to rate a variety of features for desirability on a scale of 1 to 5.  The listing below shows them ranked by 4+5 and by 3+4+5.  (The two rankings agree across the board.)


Not imp.
1


2


3


4

Imp.
5
4+5Rank3+4+5Rank
Ports of key clojure/contrib libraries
2
6
9
15
15
30
1
39
1
Sample framework interop code
5
5
10
15
12
27
2
37
2
Leiningen support for 'pure' ClojureCLR projects
5
6
9
14
12
26
3
35
3 (T)
DLR hosting (integration w/ C#, IRuby, etc.)
6
3
13
13
9
22
4
35
3 (T)
Wrappers for MS libraries/frameworks
9
6
9
15
6
21
5
30
5
Emacs support for 'pure' ClojureCLR projects
15
3
10
8
11
19
6
29
6
Visual Studio integration
12
11
9
5
11
16
7
25
7
NuGet distribution
17
6
12
6
2
8
8
20
8


This question also allowed for additional comments.  Comments included:
  • We need Clojure to be a first-class citizen on par with C#/F# before we can make the switch. This is more important for us than Visual Studio integration or the porting of libraries. A "gimped" Clojure on par with ClojureScript that was hosted on the CLR directly rather than on the DLR would be enough for us to make the switch from C#.
  • one word: mono
  • Writing my libs in Clojure.CLR and using them in ASP.NET apps.
  • Visual Studio is not that important to me personally. But none of my coworks would get theire hands dirty with vim/vimclojure
  • We need a visual studio plugin for visual studio express.  It would make it easier for people to experiment with clojure clr.
  • startup duration !!
  • What I really want is not so much Leiningen support for ClojureCLR, but to see a pure Clojure Leiningen replacement (where ""pure Clojure"" means at least ""runs on both Clojure(JVM) and ClojureCLR). 
  • With regards to VS integration, vsClojure seems OK and useful in the limited amount I have used it thus far. [...] I'd probably be all for even better VS integration. I just don't *need* it. 
  • With regards to DLR hosting, anything that makes interop between ClojureCLR and existing CLR languages easier/seamless has my vote. In my particular case, this would probably be F# and C#, in that order. IronRuby and IronPython might be useful and fun if that integration with ClojureCLR was really powerful and interesting, [...]."
  • Mono/Monodevelop support

With which Microsoft frameworks would you like to use ClojureCLR?


"Other" included:


  • MS Office
  • installation, etc.
  • Microsoft Solver Foundation
  • Kinect




If viability (long-term prospects, maturity, etc.) are a concern (for yourself or to present a case at work), what do you need to see to improve confidence?
  • A dedicated core community that is focused exclusively on the CLR (probably precludes any existing Clojure/dev or Clojure/core people?).
  • I would need to see signs of momentum: users downloading it and asking questions, reporting bugs, asking for features and chipping in, writing the odd .NET wrapper here and there, complaining about and contributing tooling, and talking/blogging about how they hope to use it, strengths and weaknesses, why it's viable, etc. Visible support and contributions from Clojure brass would go a long way to showing viability as well.
  • Very solid VS integration and interop with C#
  • community, books with .net specific samples to avoid setting up a jvm environment
  • Better visual studio support.
  • I need Rich Hickey and clojure/core to give the utmost "seal of approval" that ClojureCLR is 100% feature complete for a given version of Clojure proper. To a degree (i.e. for my boss), ClojureCLR needs to be released BY clojure/core and will not be considered until it is.
  • More info about the state of the project and more info about its integration in the mono ecosystem.
  • Larger community with wider array of active projects
  • A solid Mono story.  Some ports of major web libraries like compojure, noir, etc.

If the lack of community is an issue: What would be useful to you?

20 (63%) -- Separate ClojureCLR forum/mailing list
29 (91%) -- Blog or other forum for articles and commentary
 3 ( 9%) -- Other


For 'Other' was listed the following:

  • real endorsement from people at Clojure head
  • First-class Clojure entity.
  • Active Twitter presence


Additional comments included:
  • Im new to clojure and I love it! I work in a .net environment and I think clojure clr is the way I will get clojure into our environment. Keep clojure clr coming!
  • I think that it is good running Clojure in many platform (JVM, .NET Platform, JavaScript) and I am interested in ClojureCLR. I think that there are many Clojure programmers who use non-Windows OS. Because they cannot use Visual Studio and shoud use Mono to use ClojureCLR, so they tend to choose JVM.
  • I would love to use CLR to bypass the strange quirks I ocassionally running in using java (yes the language) on the windows platform
  • I think it's wonderful that Clojure has a port to the CLR. From a community standpoint, it seems that the "kinds of people" who would use a Clojure either (1) aren't in Windows-heavy environments or (2) would try something like F# with its VS integration and name recognition before ClojureCLR. Almost more than other Clojures, Clojure on the CLR needs a "killer app," or at least a killer wrapper/framework around existing .NET libraries to create a greater impression of activity and reliability.
  • First, if CLR is really a Clojure target, Clojure as a language should be more abstracted from JVM implementation, to allow a seemless implementation on the CLR (thus less less to worry about contrib libs porting)
  • Second, CLR should be taking as seriously as JS or JVM targets by Rich and his entourage.
  • I would love to use CLR to bypass the strange quirks I ocassionally running in using java (yes the language) on the window splatform. Just as a way to make installers and so on without having to go through java->new javalibs etc.
  • server side would be clojure JVM anyways, i'm mostly interested in client side for clojure CLR.
Thanks to all who took the time to respond.

I'll draw some conclusions and ask for input on future directions in a future posting (soon).


1 comment: