Thursday, December 29, 2011

Calling generic methods

To take advantage of some of the recent API goodness coming from Microsoft, interoperating with generic methods is a must.  Here's an example from System.Linq.Reactive.Observable:

public static IObservable<TResult> Generate<TState, TResult>( 
    TState initialState, Func<TState, bool> condition, 
    Func<TState, TState> iterate, 
    Func<TState, TResult> resultSelector, 
    Func<TState, TimeSpan> timeSelector )

and from the land of Linq, in System.Linq.Enumerable:

public static IEnumerable<TResult> GroupBy<TSource, TKey, TElement, TResult>( 
    this IEnumerable<TSource> source, 
    Func<TSource, TKey> keySelector, 
    Func<TSource, TElement> elementSelector, 
    Func<TKey, IEnumerable<TElement>, TResult> resultSelector, 
    IEqualityComparer<TKey> comparer )

Much of the goodness of Linq, Reactive Framework and others comes from the ability to chain together generic method calls with minimum specification of type arguments for those calls.   If you are in a statically-typed language such as C#, there are plenty of types floating around to do inferencing on. Of course, with dynamic, C# is not quite the paragon of static typing it once was. The mechanisms C# uses for dynamic call sites surface in the Dynamic Language Runtime and so are available to the wider world. Following the path blazed by IronPython, we have recently enhanced ClojureCLR's ability to interoperate with generic methods.

Start a REPL and start typing:

(import 'System.Linq.Enumerable)
(def r1 (Enumerable/Where [1 2 3 4 5] even?))
(seq r1)                                        ;=> (2 4)

"But of course", you say. Not so fast; under the covers there is merry mischief.

The generic method Where is overloaded with the following signatures.

public static IEnumerable<TSource> Where<TSource>(
    this IEnumerable<TSource> source,
    Func<TSource, bool> predicate)

public static IEnumerable<TSource> Where<TSource>(
    this IEnumerable<TSource> source,
    Func<TSource, int, bool> predicate)

In the Where call above, the [1 2 3 4 5] is a clojure.lang.PersistentVector. This class implements IEnumerable<Object> and so matches the IEnumerable<TSource> in the first argument position.

The value of even? is a clojure.lang.IFn, more specifically a clojure.lang.AFn. In ClojureCLR, clojure.lang.AFn implements interfaces allowing it take take part in the DLR's generic method type inferencing protocol. One interface answers queries about the arities supported by the function. The function even? reports that it supports one argument and does not support two arguments. Therefore, it supports casting to Func<TSource, bool> but not to Func<TSource, int, bool>, allowing discrimination between the two overloads. (Func<TSource, bool> is a delegate class representing a method taking one argument of type TSource and returning a value of type Boolean.)

From this information, we can pick the overload of Where to use. The value returned by the Where call is of type System.Linq.Enumerable+WhereEnumerableIterator<Object>. This type implements IEnumerable and seq can do the expected thing to it.

If we had a function f that supports one and two arguments, say

(defn f
  ([x] x)  
  ([x y] [x y]))

the type inferencing illustrated in the previous example would not work.

(Enumerable/Where [1 2 3 4 5] f)   ;=> FAIL!
The error messages is quite clear:

ArgumentTypeException Multiple targets could match:  
  Where(IEnumerable`1, Func`2),
  Where(IEnumerable`1, Func`3)

There are two ways around this. The simplest is to use an anonymous function of one argument that calls f:

(Enumerable/Where [1 2 3 4 5] #(f %1))

The second way is to declare the types explicitly. Macros sys-func and sys-action are available to create a function with delegate type matching  System.Func<,...> and System.Action<,...>, respectively.

(Enumerable/Where [1 2 3 4 5] (sys-func [Object Boolean] [x] (f x))))

The first way clearly is preferable. However, when type inferencing does not suffice, sys-func and sys-action can be used. (If delegates of types other then Func<,...> and Action<,...> are required, gen-delegate is available.)

Not just Clojure data structures can participate as sources:

(def r2 (Enumerable/Range 1 10))
(seq r2) ;=> (1 2 3 4 5 6 7 8 9 10)
(seq (Enumerable/Where r2 even?)) ;=> (2 4 6 8 10) 

In fact, any of the following calls will work:

(Enumerable/Where r2 (sys-func [Int32 Boolean] [x] (even? x)))
(Enumerable/Where r2 (sys-func [Object Boolean] [x] (even? x)))
(Enumerable/Where (seq r2) even?)

There are situations where you need to supply type arguments explicitly to a generic method. For example, the following fails:

(def r3 (Enumerable/Repeat 2 5) ;=> FAILS!

The error message states:

InvalidOperationException Late bound operations cannot be performed 
on types or methods for which ContainsGenericParameters is true.

We can cause the type arguments on the Repeat<T> method to be filled in using the type-args macro:

(def r3 (Enumerable/Repeat (type-args Int32) 2 5))
(seq r2) ;=> (2 2 2 2 2)

If you'd like to do Linq-style method concatenation, don't forget the threading macro:

(seq (-> (Enumerable/Range 1 10)
            (Enumerable/Where even?)
            (Enumerable/Select #(* %1 %1))))  ;=> (4 16 36 64 100)

Of course, if you'd like to have the Linq syntax that is available in C#, you are free to write a macro.

Tuesday, December 27, 2011

Working with enums


Working with enum types is straightforward. We have provided a few core methods to simplify certain operations.

Accessing/creating enum values

An enum type is a value type derived from System.Enum. The named constants in an enum type are implemented as static fields on the type.  For example, System.IO.FileMode, defined in C# as

public enum FileMode
{
    Append = 6,
    Create = 2,
    CreateNew = 1,
    Open = 3,
    OpenOrCreate = 4,
    Truncate = 5
}

has an MSIL implementation more-or-less equivalent to

[Serializable]
public sealed class FileMode : System.Enum
{
    public static System.IO.FileMode CreateNew = 1;
    public static System.IO.FileMode Create = 2;
    public static System.IO.FileMode Open = 3;
    public static System.IO.FileMode OpenOrCreate 4;
    public static System.IO.FileMode Truncate = 5;
    public static System.IO.FileMode Append = 6;
}

Thus, we can use our regular static-field access interop syntax to retrieve these values:

(import 'System.IO.FileMode) ;=> System.IO.FileMode
FileMode/CreateNew           ;=> CreateNew

These are not integral-type values. They retain the enumeration type.

(class FileMode/CreateNew) ;=> System.IO.FileMode

You can convert them to an integer value if you desire:

(int FileMode/CreateNew) ;=> 1

If you want to convert from an integer value to an enumeration value, try:

(Enum/ToObject FileMode 4) ;=> OpenOrCreate

If you want convert from the name of an integer to an enumeration value, the enum-val method will work with strings or anything that name works on:

(enum-val FileMode "CreateNew") ;=> CreateNew
(enum-val FileMode :CreateNew)  ;=> CreateNew


Working with bit fields

Enumeration types that have the Flags attribute are often used to represent bit fields. For convenience, we provide methods bit-or and bit-and to to combine and mask bit field values. For example, System.IO.FileShare has the Flags attribute. It is defined as follows:

[Serializable, ComVisible(true), Flags]
public enum FileShare
{
 Delete = 4,
 Inheritable = 0x10,
 None = 0,
 Read = 1,
 ReadWrite = 3,
 Write = 2
}

Use enum-or to combine values.

(import 'System.IO.FileShare)
(enum-or FileShare/Read FileShare/Write) ;=> ReadWrite

Use enum-and to mask values.

(def r (enum-or FileShare/ReadWrite FileShare/Inheritable))
(= (enum-and r FileShare/Write) FileShare/Write) ;=> true
(= (enum-and r FileShare/Write) FileShare/None)  ;=> false
(= (enum-and r FileShare/Delete) FileShare/None) ;=> true


You can also use the HasFlag method to test if a bit is set:

(.HasFlag r FileShare/Write)  ;=> true
(.HasFlag r FileShare/Delete) ;=> false


Monday, December 26, 2011

Using ngen to improve ClojureCLR startup time

Startup speed ranked fairly low in importance in the 2011 ClojureCLR survey.  Still, it can be annoying and is an impediment to certain uses of ClojureCLR.

I've used several profiling tools to examine the startup period.   The only conclusion I've been able to draw:  JIT-compilation is the culprit.  The percentage of startup time devoted to JITting is in excess of 90%.  One solution to this: pre-JIT.

If you run ngen.exe on the primary DLLs involved in ClojureCLR startup, you will experience significant startup time improvement.  I ran

time ./Clojure.Main.exe -e  "(println :a)"

on a 4.0 Debug build, on a 4.0 Release build, and on a 4.0 Release build with ngen.  For comparison, I also ran
 time java -jar clojure.jar -e "(println :a)"
(I used my git-bash shell via msysgit, so time was available via mingw.)

Results:

DebugReleaseRelease / ngenJVM
real
0m5.118s
0m8.130s
0m0.231s
0m1.053s
user
0m0.000s
0m0.015s
0m0.000s
0m0.000s
sys
0m0.000s
0m0.000s
0m0.015s
0m0.000s

The slower startup time of Debug vs non-ngen'd Release is no doubt due to the more extensive JIT optimizations taking place in the latter.  Approximately four times as fast as the JVM version is good enough for me.

I did the following ngens.  All these DLLs are loaded on an intial startup through one eval and printing.

ngen install Clojure.dll
ngen install clojure.clr.io.clj.dll
ngen install clojure.core.clj.dll
ngen install clojure.core.protocols.clj.dll
ngen install clojure.core_clr.clj.dll
ngen install clojure.core_deftype.clj.dll
ngen install clojure.core_print.clj.dll
ngen install clojure.core_proxy.clj.dll
ngen install clojure.genclass.clj.dll
ngen install clojure.gvec.clj.dll
ngen install clojure.main.clj.dll
ngen install clojure.pprint.cl_format.clj.dll
ngen install clojure.pprint.clj.dll
ngen install clojure.pprint.column_writer.clj.dll
ngen install clojure.pprint.dispatch.clj.dll
ngen install clojure.pprint.pprint_base.clj.dll
ngen install clojure.pprint.pretty_writer.clj.dll
ngen install clojure.pprint.print_table.clj.dll
ngen install clojure.pprint.utilities.clj.dll
ngen install clojure.repl.clj.dll
ngen install clojure.walk.clj.dll

You will get errors, mostly of the form
1>Common Language Runtime detected an invalid program. while compiling method clojure/walk$macroexpand_all$fn__12034__12039..ctor

It works nevertheless.  Someday I'll have to figure out exactly what sin I'm committing in my constructor code.

I cannot competently hypothesize why ClojureCLR kills the JITter like this, in comparison to the JVM, or in comparison to other similar sized programs.  Delayed JITting may be one reason.  Also, the generated ClojureCLR code contains twice as many classes as the JVM code does, due to the tactic I use to workaround the inability of the DLR to generate instance methods.


Saturday, December 24, 2011

ClojureCLR has a new home

The survey action items had as item #1 the following:
Action item: Ask Rich Hickey and Clojure/core to clarify their position and/or plans re ClojureCLR.
You can refer to the post on viability to to get a better sense of what responders were looking for.

As a result of a discussions with core, ClojureCLR has moved to a better neighborhood.  The repo has moved under the clojure group at github:
https://github.com/clojure/clojure-clr
joining the mainline Clojure project, ClojureScript and the contrib libs.

ClojureCLR is now an official project under dev.clojure.org:
http://dev.clojure.org/jira/browse/CLJCLR
Issues will be managed through this site.   We will continue to maintain RHCAH: Rich Hickey Contributor Agreement Hygiene.  Please have a CA on file with Rich before submitting patches.

I've used the move as an opportunity to rewrite and reorganize the wiki pages.  I'll be posting entries here on some of the newest material, such as the improved support for generic method interop.

This (much appreciated) token of core's regard is likely to be the extent of official support for ClojureCLR  for the time being.  Core does not have sufficient resources for more aggressive stewardship of the project.

The answer to the action item: ClojureCLR is a community-supported project.  If you are interested in the long-term success of Clojure on the CLR, roll up your sleeves.






Thursday, November 3, 2011

Survey says: be eco-friendly

I included a number of questions on the survey that related to the environment.  For code development, that is. I had in mind two development scenarios:


  1. "Pure" clojure development: these are projects that are Clojure-centric.  This could be a simple port of Clojure project to the CLR, or any Clojure project where Visual Studio support is not critical, i.e., not involving co-development of C# code or the use of big frameworks such as ASP.NET.  Such projects could easily be developed using the same tools as JVM Clojure developers.  I used Leiningen and Emacs as examples for project management and source editing.
  2. Clojure projects where Visual Studio capabilities might be crucial, such as ASP.NET or WPF projects, or just heavy development of C# and Clojure code together.

Leiningen/Emacs support dominated Visual Studio integration, NuGet, etc.

In the frustrations/deterrents section:

#8 -- Availability of Clojure-CLR targeted editors/dev environments
#10 -- Lack of integration with Visual Studio

In the desirability section:

#3 -- Leiningen support for 'pure' ClojureCLR projects
#6 -- Emacs support for 'pure' ClojureCLR projects
#7 -- Visual Studio integration
#8 -- NuGet distribution

Even Mono got higher ranking or more mention than VS.  However, some mentioned that lack of VS integration has been a show-stopper.

Action item: Develop a version of Leiningen supporting ClojureCLR projects (nlein?) 
Action item: Develop a NuGet distribution for ClojureCLR, and perhaps ported contrib libs
Action item: Develop Emacs support for ClojureCLR
Action item: Improve VS integration (vsClojure?)

I'm fairly comfortable taking a look at creating an 'nlein' and working NuGet distributions (though others should feel free to step in).  However, I have no interest in IDE development, whether hacking Emacs or developing VS plug-ins.  Others are going to have to lead those efforts.

Mono compatibility probably has been the most frequent question on the Clojure ML and on IRC.  It seems to matter to an important subset of potential users.  I've not wanted to deal with yet another development environment.  I hope someone will step up to this one before I have to.
Action item: Investigate Mono compatibility

Survey says: interoperate

One thread running through the survey had to do with interoperability and the specifics of working with Microsoft frameworks.

"Availability of ClojureCLR-specific documentation" was fourth on the list of frustrations.  "Interop support" was sixth. Under desirability, "Sample framework interop code" was #2 and "Wrappers for MS libraries/frameworks" was # 5.

Related comments include:
  • In general the lack of examples of typical uses, something which is pretty well covered in the JVM based Clo[j]ure, things like jetty/ring, jdbc, and GUI (which tends to be a lot more important in the windows world).
  • Support for .net generics is extremely important! Lack of [this and another] features were the show stoppers for me. 
  • Writing my libs in Clojure.CLR and using them in ASP.NET apps.
  • anything that makes interop between ClojureCLR and existing CLR languages easier/seamless has my vote.
  • interop with C#
The survey asked respondents to indicate MS frameworks of interest.  Those receiving more than one vote:

Linq 26 59%
ASP.NET / MVC20 45%
Reactive Framework15 34%
WPF13 30%
Windows Runtime (Win8) 10 23%
WinForms 9 20%
XNA (incl. XBox) 9 20%
Silverlight  7 16%
WCF  7 16%
Azure  7 16%
Entity Framework 5 11%
Robotics Studio 5 11%
Sharepoint  3  7%

Linq's #1 status surprised me.  I think of Linq as providing a functional approach to accessing sequential data, and Clojure is ... that.  However, my use of Linq to date is minimal--implementing ClojureCLR does not stretch that muscle.  Perhaps people want to write Linq extensions in Clojure for use in C#?  Perhaps some could comment on this.

Question for you:  What form of Linq interop is desired?  

The next several entries--ASP.NET/MVC, Rx, WPF--did not surprise me.  Reading it just now, I realized that I left out the TPL (Task Parallel Library).  I wonder where that would have ranked.

I tossed in Windows Runtime (WinRT from Win8) for fun.  Its high rank was a surprise. 
Question for you: How are you planning to use ClojureCLR with WinRT?
Steps suggested by the data:
Action item: Provide better documentation and examples for interop at the class/method level, particularly for things not in the Java world, such as true generic classes and methods, by-ref parameters, etc.
 Action item: Provide tutorial examples of interop with key MS frameworks: Linq, ASP.NET/MVC, Reactive Framework, WPF, TPL.
The situation regarding wrappers for MS libraries and frameworks is less clear to me.  You can't 'wrap' ASP.NET.   Calling out to ClojureCLR code from ASP.NET seems more likely.  Interop from C# back to ClojureCLR is the real problem here (probably).  I can imagine wrappers for smaller things such as Rx, TPL, etc.
Action item: Provide tutorial examples of C# calling back into ClojureCLR.
Action item: Identify smaller libraries that would benefit from wrappers and implement same. 

Survey says: port those libs



Clojure/contrib lib ports

I suspected this would be high on the list, but hitting #3 was a surprise.  This was one community task I thought people might get drawn to first.   For the most part, it's not that hard to do, and there are significant examples: all the files such as clojure/core.clj, gvec.clj, and xml.clj  in the main ClojureCLR distribution are ported libs. However, as one comment stated:
I haven't even really seen anything in terms of guidance / best practices to allow this work to be carried forward by others in a consistent manner.
 Okay, here it is:
Replace interop calls to Java methods with interop calls to CLR methods.
That covers most of it. I've managed to keep the port of clojure/core.clj to inline changes across almost 6000 lines of code.   However, more accessible examples might be useful.
Action item: Provide ports of key libs to 'prime the pump'.
Action item: Illustrate the porting process and provide more detailed guidelines in a future post.
The ongoing reorganization of the contrib libs will be big help in moving this forward.  It makes the mass of clojure/contrib more modular and gives clues to what are viable libraries.

The biggest problem  is not the effort to port but how to manage the code when done.  Ideally, the libs would contain JVM and CLR versions in one code base.  There are ways to write *.clj code that would lessen the burden of porting.  However, I don't see contrib lib authors taking on that burden.  And the mind-set is not there from the top:  Rich has been very clear on two points: there will be no conditionalized read mechanism; and that Clojure code involving interop is never intended to be portable.

So, at the moment, it appears that parallel projects, with 90% duplicate code, is the only option.  Duplication carries the burden of maintaining consistency.  
Action item: Find a way to manage maintenance of ported libs.
Which contrib libs should be done first?

Survey says: we need community



I think one respondent captured the problem quite well:
I would need to see signs of momentum: users downloading it and asking questions, reporting bugs, asking for features and chipping in, writing the odd .NET wrapper here and there, complaining about and contributing tooling, and talking/blogging about how they hope to use it, strengths and weaknesses, why it's viable, etc.
 Yeah. Me, too.

Community building takes effort.  In the beginning, I hoped that if I just took care of questions coming up on the mailing list, the rest would develop with a few hardy individuals starting to contribute.    I didn't (and still don't) have time to hang on IRC.  I was usually behind bringing ClojureCLR up-to-date with Clojure and felt that the best thing was for ClojureCLR to be as close to feature-complete as I could make it.  Hence, I neglected the community building.

I asked two questions on the survey about specific community activities:
  • Separate ClojureCLR forum/mailing list 
  • Blog or other forum for articles and commentary 
Twenty of 53 respondents checked the ML question;  29 of 53 indicated a blog or similar is desired.
Action item: Now that there is blog (here!), use it to inform and educate the community.
The comments on the survey give some ideas on what would be useful:  best practices, examples of interop, progress reports, roadmaps, etc.

Please use the comments to give input on what you'd like to see.

Several people had suggested prior to the survey that people primarily interested in CLR might be put off by the high volume of irrelevant traffic on the Clojure ML.  So I put in the question on forming a separate ML for ClojureCLR.  I hesitate to create a mailing list at this time.  Nothing says "not viable" like a mailing list that has no activity.  Is this something that should wait until there is more of a community need?
I'd love to hear what others think will be effective in building the ClojureCLR community.

Survey says: viability is number one


I'm not surprised viability is the #1 concern.  This concen boils down to:
  1. ClojureCLR has no official status.
  2. ClojureCLR has no community.
ClojureCLR is obviously pretty much a one-person operation.  The project's relationship to the overall Clojure effort is not clear.  There is one mention on the home page of clojure.org.  If you search for 'CLR' on dev.clojure.org, you will get three hits: one for an article stub for installing on CLR/VS (I didn't know that existed until just now); one mention as a possible backend for CinC; and a mention on the JIRA workflow page.  Since the JIRA workflow sends all requests through Clojure/core members that bring us back to the 'no official status':  Clojure/core does not appear to acknowledge the existence of ClojureCLR.  

For some, the lack of official status is enough by itself to preclude any use of ClojureCLR.  The comments for "what do you need to see to improve confidence [in viability]" include:
  • Visible support and contributions from Clojure brass would go a long way to showing viability as well.
  • I need Rich Hickey and clojure/core to give the utmost "seal of approval" that ClojureCLR is 100% feature complete for a given version of Clojure proper. To a degree (i.e. for my boss), ClojureCLR needs to be released BY clojure/core and will not be considered until it is.
  • A dedicated core community that is focused exclusively on the CLR (probably precludes any existing Clojure/dev or Clojure/core people?).
  • CLR should be taking as seriously as JS or JVM targets by Rich and his entourage.
  • real endorsement from people at Clojure head
Out-of-band, I've received additional comments that the absence of ClojureCLR from dev.clojure.org and the fact that the ClojureCLR repo did not move from github.com/richhickey to github.com/clojure along with Clojure and the contribs (and now also ClojureScript) are at least highly suspect.

There's not much I can do about this concern, except to:
Action item: Ask Rich Hickey and Clojure/core to clarify their position and/or plans re ClojureCLR.
The answer to this question could affect the other actions items, but I won't second-guess that.

As for (2), community, that deserves its own post.

Survey says: a call to action

I created the 2011 ClojureCLR Survey to get some guidance on moving ClojureCLR forward. The survey results give pretty clear indication of what needs to be worked on to move ClojureCLR forward.

The writeup is a bit long.  So that the whole thing is  not just "tl;dr" for the masses, I'll split it into multiple posts.

Spoiler alert: Here are all the action items suggested in the various posts.

Action item: Ask Rich Hickey and Clojure/core to clarify their position and/or plans re ClojureCLR.

Action item: Now that there is blog (here!), use it to inform and educate the community.

Clojure/contrib lib ports 
Action item: Provide ports of key libs to 'prime the pump'.
Action item: Illustrate the porting process and provide more detailed guidelines.
Action item: Find a way to manage maintenance of ported libs.


Action item: Provide better documentation and examples for interop at the class/method level, particularly for things not in the Java world, such as true generic classes and methods, by-ref parameters, etc.
 Action item: Provide tutorial examples of interop with key MS frameworks: Linq, ASP.NET/MVC, Reactive Framework, WPF, TPL.
Action item: Provide tutorial examples of C# calling back into ClojureCLR. 

Action item: Identify smaller libraries that would benefit from wrappers and implement same. 

Ecosystem 
Action item: Develop a version of Leiningen supporting ClojureCLR projects (nlein?) 
Action item: Develop a NuGet distribution for ClojureCLR, and perhaps ported contrib libs
Action item: Develop Emacs support for ClojureCLR
Action item: Improve VS integration (vsClojure?)
Action item: Investigate Mono compatibility 

Does this look like a reasonable agenda for the immediate future of ClojureCLR development.  The lines are open and agents are standing by to take your call.




Tuesday, November 1, 2011

Results of the 2011 ClojureCLR Survey

The 2011 ClojureCLR Survey concluded last night.  Inspired by Chas Emerick's State of Clojure Surveys (2010, 2011), the survey was an attempt to get feedback from those interested in ClojureCLR regarding usage, problems, needs and desires, in the hope of setting the direction for ClojureCLR development.

The survey responses are summarized below.  I will provide my analysis in a later posting.

The survey was announced by three posts to the Clojure mailing list and two tweets from my Twitter account.  Since I had never tweeted before (I may be anti-social), I made sure a few select people were following so that there would be minimum of retweeting.

There were 53 responses to the survey.  This number is 8% of the 670 responses to the 2011 State of Clojure survey.  For comparison, the ClojureCLR repo has 29% the number of watchers of the Clojure repo.  There is no way to know how representative this sample in the universe of Clojure folk.

Make of this what you will:  Given ten days to respond,  36% of the responses came on the last day of the survey.  :)

The first two questions on the survey were an attempt to judge the respondent's past and present engagement with ClojureCLR.


What is your level of experience with ClojureCLR?




27 (51%) -- Never met
21 (40%) -- Dated a few times
 2 ( 4%) -- Hanging out
 2 ( 4%) -- Entertaining thoughts about a future together
 1 ( 2%) -- Raising a family
The majority of respondents have never used ClojureCLR.  Just 9% have used it more than a few times.


How would you characterize your use of ClojureCLR today?
 


27 (51%) -- Really, I've never touched it.
16 (30%) -- I tried it, gave it up.
 8 (15%) -- I'm just tinkering.
 2 ( 4%) -- I use it for serious "hobby" projects.
 0 ( 0%) -- I use it at work. 


"Never met" was almost perfectly correlated with "Really, I've never touched it", as it should be. "Dated a few times" was unfortunately mostly paired with "I tried it, gave it up". (Some of the other pairings suggest I should have been less cute with the labels.)

Never
touched
Tried it
gave it up
Tinkering Serious
"hobby"
Work
Never met
25
1
1
Dated a few times
1
15
5
Hanging out
1
1
Thoughts of a
future together
1
1
Raising a family
1

At any rate, it is clear that the level of involvement is very low overall.


Which of the following apply to your Microsoft development experience?



14 (27%) -- I presently develop on the .Net platform for non-work projects.
27 (52%) -- I presently develop on the .Net platform at work.
26 (50%) -- I have tried other non-Microsoft languages targeting the .Net platform.
15 (29%) -- My workplace does not use the .Net platform.
16 (31%) -- My workplace allows non-Microsoft languages targeting the .Net platform.
 8 (15%) -- My workplace permits ClojureCLR for development.
 8 (15%) -- My workplace would never permit ClojureCLR for development. 



What has been most frustrating for you in your use of ClojureCLR or kept you from using it (more or at all)? (Over and above general Clojure issues.)

This question asked the respondent to rank each of a set of categories on a 1-5 scale of importance.

Not  imp.
1

2

3

4
Very imp.
5
Availability of ClojureCLR-specifc documentation
5
8
11
8
13
ClojureCLR's install process
8
4
12
7
12
Difficulty deploying completed applications
11
8
12
5
4
Clojure/contrib libs not ported
5
4
11
11
11
Lack of integration with Visual Studio
14
6
9
7
7
Availability of ClojureCLR-targeted editors/dev environments
10
8
9
6
10
Startup speed
13
8
7
4
6
Runtime performance
9
6
10
9
5
Interop support
5
8
8
9
11
Concern about lack of community
3
3
14
11
13
Concern about quality
10
7
6
6
14
Concern about long-term viability
6
3
5
13
16
Mono
13
5
6
6
11

Focusing on the  total number of responses of 4+5 or 3+4+5 is one way to get a ranking of relative importance.

3+4+5Rank4+5Rank
Concern about lack of community
38
1

24
2
Concern about long-term viability
34
2

29
1
Clojure/contrib libs not ported
33
3

22
3
Availability of ClojureCLR-specifc documentation
32
4

21
4
ClojureCLR's install process
31
5

19
7
Interop support
28
6

20
5 (T)
Concern about quality
26
7

20
5 (T)
Availability of ClojureCLR-targeted editors/dev environments
25
8

16
9
Runtime performance
24
9

14
10 (T)
Mono
23
10 (T)

17
8
Lack of integration with Visual Studio
23
10 (T)

14
10 (T)
Difficulty deploying completed applications
21
12

9
13
Startup speed
17
13

10
12

The top four in each ranking contain the same items.  It is possible that people who tried ClojureCLR early and were disappointed in the install process and interop support would be happier with these in the most recent version;  this deserves investigation.  Because I suspected that the questions about community and viability would be high on the list, later questions probed these topics more.  See below.

This question allowed additional comments.  Among the deterents noted:
  • In general the lack of examples of typical uses, something which is pretty well covered in the JVM based Clo[j]ure, things like jetty/ring, jdbc, and GUI (which tends to be a lot more important in the windows world).
  • Emacs support as good as standard clojure's is extremely important! Support for .net generics is extremely important! Lack of these two features were the show stoppers for me. 
  • Lack of ported libraries is also pretty important.  
  • I haven't even really seen anything in terms of guidance / best practices to allow this work to be carried forward by others in a consistent manner.
  • Also, it would be great if there were more readily available progress reports, roadmaps, etc from David et al.  Until this survey became available, ClojureCLR hasn't had a blog, so that's a good start.
  • VsClojure is unreliable. Trying to build the self generated main() works maybe one or two times, then keeps failing when the bootstrap compiler aborts. I guess this might be covered by the vs integration question but it turns off developers immediately. 
  • Microsoft can kill off support at any time. Linux is a 2nd class citizen. .NET/CLR is not a viable target for anything.
  • The completeness of the ClojureCLR implementation compared to the Clojure implementation. I.e. do they do the same thing? Is there a Clojure Spec suite of tests or something like that that both implementations pass?
  • Lack of personal need for CLR.
  • Generally the install seems bulky compared to clojure for the jvm.
  • I hadn't heard of it. I'm quite interested in Clojure, but was not aware of a CLR port. So greatest impediment is lack of knowledge/lack of PR.
  •  Mono is my insurance policy, so I want at a minimum as much .NET code as I write (to a maximum of ""all .NET code I write and use"") to run on Mono as well on MS's CLR. [...] I'd love to have ClojureCLR work seamlessly on Mono.
Rate the following for desirability.
Respondents were asked to rate a variety of features for desirability on a scale of 1 to 5.  The listing below shows them ranked by 4+5 and by 3+4+5.  (The two rankings agree across the board.)


Not imp.
1


2


3


4

Imp.
5
4+5Rank3+4+5Rank
Ports of key clojure/contrib libraries
2
6
9
15
15
30
1
39
1
Sample framework interop code
5
5
10
15
12
27
2
37
2
Leiningen support for 'pure' ClojureCLR projects
5
6
9
14
12
26
3
35
3 (T)
DLR hosting (integration w/ C#, IRuby, etc.)
6
3
13
13
9
22
4
35
3 (T)
Wrappers for MS libraries/frameworks
9
6
9
15
6
21
5
30
5
Emacs support for 'pure' ClojureCLR projects
15
3
10
8
11
19
6
29
6
Visual Studio integration
12
11
9
5
11
16
7
25
7
NuGet distribution
17
6
12
6
2
8
8
20
8


This question also allowed for additional comments.  Comments included:
  • We need Clojure to be a first-class citizen on par with C#/F# before we can make the switch. This is more important for us than Visual Studio integration or the porting of libraries. A "gimped" Clojure on par with ClojureScript that was hosted on the CLR directly rather than on the DLR would be enough for us to make the switch from C#.
  • one word: mono
  • Writing my libs in Clojure.CLR and using them in ASP.NET apps.
  • Visual Studio is not that important to me personally. But none of my coworks would get theire hands dirty with vim/vimclojure
  • We need a visual studio plugin for visual studio express.  It would make it easier for people to experiment with clojure clr.
  • startup duration !!
  • What I really want is not so much Leiningen support for ClojureCLR, but to see a pure Clojure Leiningen replacement (where ""pure Clojure"" means at least ""runs on both Clojure(JVM) and ClojureCLR). 
  • With regards to VS integration, vsClojure seems OK and useful in the limited amount I have used it thus far. [...] I'd probably be all for even better VS integration. I just don't *need* it. 
  • With regards to DLR hosting, anything that makes interop between ClojureCLR and existing CLR languages easier/seamless has my vote. In my particular case, this would probably be F# and C#, in that order. IronRuby and IronPython might be useful and fun if that integration with ClojureCLR was really powerful and interesting, [...]."
  • Mono/Monodevelop support

With which Microsoft frameworks would you like to use ClojureCLR?


"Other" included:


  • MS Office
  • installation, etc.
  • Microsoft Solver Foundation
  • Kinect




If viability (long-term prospects, maturity, etc.) are a concern (for yourself or to present a case at work), what do you need to see to improve confidence?
  • A dedicated core community that is focused exclusively on the CLR (probably precludes any existing Clojure/dev or Clojure/core people?).
  • I would need to see signs of momentum: users downloading it and asking questions, reporting bugs, asking for features and chipping in, writing the odd .NET wrapper here and there, complaining about and contributing tooling, and talking/blogging about how they hope to use it, strengths and weaknesses, why it's viable, etc. Visible support and contributions from Clojure brass would go a long way to showing viability as well.
  • Very solid VS integration and interop with C#
  • community, books with .net specific samples to avoid setting up a jvm environment
  • Better visual studio support.
  • I need Rich Hickey and clojure/core to give the utmost "seal of approval" that ClojureCLR is 100% feature complete for a given version of Clojure proper. To a degree (i.e. for my boss), ClojureCLR needs to be released BY clojure/core and will not be considered until it is.
  • More info about the state of the project and more info about its integration in the mono ecosystem.
  • Larger community with wider array of active projects
  • A solid Mono story.  Some ports of major web libraries like compojure, noir, etc.

If the lack of community is an issue: What would be useful to you?

20 (63%) -- Separate ClojureCLR forum/mailing list
29 (91%) -- Blog or other forum for articles and commentary
 3 ( 9%) -- Other


For 'Other' was listed the following:

  • real endorsement from people at Clojure head
  • First-class Clojure entity.
  • Active Twitter presence


Additional comments included:
  • Im new to clojure and I love it! I work in a .net environment and I think clojure clr is the way I will get clojure into our environment. Keep clojure clr coming!
  • I think that it is good running Clojure in many platform (JVM, .NET Platform, JavaScript) and I am interested in ClojureCLR. I think that there are many Clojure programmers who use non-Windows OS. Because they cannot use Visual Studio and shoud use Mono to use ClojureCLR, so they tend to choose JVM.
  • I would love to use CLR to bypass the strange quirks I ocassionally running in using java (yes the language) on the windows platform
  • I think it's wonderful that Clojure has a port to the CLR. From a community standpoint, it seems that the "kinds of people" who would use a Clojure either (1) aren't in Windows-heavy environments or (2) would try something like F# with its VS integration and name recognition before ClojureCLR. Almost more than other Clojures, Clojure on the CLR needs a "killer app," or at least a killer wrapper/framework around existing .NET libraries to create a greater impression of activity and reliability.
  • First, if CLR is really a Clojure target, Clojure as a language should be more abstracted from JVM implementation, to allow a seemless implementation on the CLR (thus less less to worry about contrib libs porting)
  • Second, CLR should be taking as seriously as JS or JVM targets by Rich and his entourage.
  • I would love to use CLR to bypass the strange quirks I ocassionally running in using java (yes the language) on the window splatform. Just as a way to make installers and so on without having to go through java->new javalibs etc.
  • server side would be clojure JVM anyways, i'm mostly interested in client side for clojure CLR.
Thanks to all who took the time to respond.

I'll draw some conclusions and ask for input on future directions in a future posting (soon).


Saturday, October 22, 2011

The 2011 ClojureCLR survey is open

Inspired by Chas Emerick's State of Clojure survey,  I've created a similar survey to assess the state of the ClojureCLR community and to gather ideas for moving ClojureCLR forward.

The survey is at:  


https://docs.google.com/spreadsheet/viewform?formkey=dDE3YnlYbXBxc3ljVU80bjNnTVZaMlE6MQ

The survey will remain open through October 31, 2011.


The first code commit to the ClojureCLR github site occurred on February 1, 2009. Today, ClojureCLR is a 99% feature-complete implementation of Clojure. It closely tracks developments in the JVM implementation. It is current with the 1.3.0  release (and has been current and mostly feature-complete since at least 1.1, in January, 2010).

Apparently, there is some interest in ClojureCLR.  The binary distribution of the 1.2.0 version has been downloaded 772 times.  The repo has 267 watchers, 29% of the number of watchers on the main Clojure repo.

Despite those measures, it is hard to get a sense of use, community or momentum for ClojureCLR.  The few queries on IRC often go unanswered; the mailing list sees no mention; there are no bug reports; and there is almost no sign of people asking for features or pitching in to create something simple like wrapper code or ports of the contrib libs.

What will help create that community?  What will increase confidence in the viability of this port of Clojure?  I hope this survey will give some guidance for moving ClojureCLR forward.