26 June 2012

FsCheck 0.8.1: The ecosystem release

Yesterday I’ve released a new version of FsCheck. There are some minor changes, additions and bug fixes to FsCheck itself. The most important part of this release is that FsCheck is now integrated with Xunit through the new FsCheck.Xunit assembly.

For the impatient: NuGet packages for FsCheck and FsCheck.Xunit

Here goes a more detailed overview.

Generators are now applicative. Or, without the general abstract nonsense: say you’ve got three generators

let ga,gb,gc :Gen<_> * Gen<_> * Gen<_>

And you have a function of three arguments that you’d like to map over those.

let f = fun a b c ->

No problem, you can use Gen.map3 – right? But how many instances of map do you need? FsCheck currently defines 6. But now that Gen is applicative, we can write:

let result = f <!> ga <*> gb <*> gc

Hey, doesn’t that almost read like function application? That’s the idea. The weird operators in between just make sure everything is applied properly. Besides being a bit nicer to use sometimes, you can also keep chaining this indefinitely, up to any arity.

And hey, if you still prefer map, FsCheck still has those.

New generators. Mauricio Scheffer contributed int64 and TimeSpan generators. I’ve also added an int16 generator, and DontSize wrappers for all of those – the default generators for int16, 32 and 64 will generate integers smaller than the test size, which is increased by FsCheck as the test progresses. The DontSize variants ignore the size and always pick from the whole range for the type.

Xunit integration. Xunit is a very nice framework – very simple to use and extend. FsCheck integrates with Xunit by adding a PropertyAtttribute. To use it, just use Property instead of Fact, and you can now give your test arguments. These are randomly generated by FsCheck. You can also use all of the normal property combinators in the Prop namespace. A couple of examples:

[<Property>]
let ``abs(v) % k equals abs(v % k)`` v (NonZeroInt k) = 
    (abs v) % k = abs(v % k)
[<Property>]
let ``divMod should satisfy definition`` (x:int) (y:int) = 
    y <> 0 ==> lazy (let (d,m) = divMod x y in d*y + m = x)

Note that Property test methods, as opposed to Fact, don’t need to return unit – they can return anything that’s testable with FsCheck, including bool. But since a test fails in FsCheck when it throws an exception, you can still use the Assert class if you like. You can also use FsUnit or Unquote or whatever to write your assertions. FsCheck does not care – it does the hard work of generating random values and shrinking them if the test fails and generally tries to get out of your way for the rest.

You can also set various configuration parameters using the Property attribute, which then apply to that method only. Most of these are trivial (e.g. the number of tests), but an interesting one is that you can override the random generation for certain types on one test method only:

type TestArbitrary1 =
    static member PositiveDouble() =
        Arb.Default.Float()
        |> Arb.mapFilter abs (fun t -> t >= 0.0)

[<Property( Arbitrary=[| typeof<TestArbitrary1> |] )>]
let ``should register Arbitrary instances from Config in last to first order``(underTest:float) =
    underTest >= 0.0

Here the generator for float is overridden to generate only positive floats.

Finally, a special thanks to @mausch for contributing, integrating FsCheck with Fuchu, and giving me a gentle nudge to get this release out the door at last!

Technorati Tags: ,,
Share this post : MSDN! Technet! Del.icio.us! Digg! Dotnetkicks! Reddit! Technorati!

3 comments:

  1. Kurt, I got FsCheck.Xunit w/ NuGet, but it looks like it's referring to an old version of FsCheck. Trying to use [Property] gives an exception.

    (Next move: I'll go check if you've posted the source code anywhere so I can rebuild vs. the latest FsCheck.)


    System.IO.FileLoadException: Could not load file or assembly 'FsCheck, Version=0.8.0.0, Culture=neutral, PublicKeyToken=465a4f97aa0d19b7' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
    File name: 'FsCheck, Version=0.8.0.0, Culture=neutral, PublicKeyToken=465a4f97aa0d19b7'
    at FsCheck.PropertyAttribute..ctor()
    at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor)
    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
    at Xunit.Sdk.Reflector.ReflectionMethodInfo.d__0.MoveNext()
    at Xunit.Sdk.Executor.EnumerateTests..ctor(Executor executor, Object _handler)
    at Xunit.ExecutorWrapper.RethrowWithNoStackTraceLoss(Exception ex)
    at Xunit.ExecutorWrapper.CreateObject(String typeName, Object[] args)
    at Xunit.ExecutorWrapper.EnumerateTests()
    at Xunit.TestAssemblyBuilder.Build(IExecutorWrapper executorWrapper)
    at Xunit.MultiAssemblyTestEnvironment.Load(IExecutorWrapper executorWrapper)
    at Xunit.ConsoleClient.Program.RunProject(XunitProject project, Boolean teamcity, Boolean silent)
    at Xunit.ConsoleClient.Program.Main(String[] args)

    ReplyDelete
  2. Follow-up: the binaries you posted on CodePlex don't have the same issue as the NuGet package.

    Solution: download http://fscheck.codeplex.com/downloads/get/125486

    Thanks Kurt, this looks really cool!

    ReplyDelete
  3. Thanks for letting me know. I've now uploaded FsCheck 0.8.2 and FsCheck.Xunit 0.2 that should fix this problem. I thought it would be a good idea to sign the dlls, but then you run into these shenanigans, so I made them unsigned again.

    ReplyDelete