Handling test data clean up in SpecFlow

Writing advanced UI tests with SpecFlow and Selenium often requires some test data to be created. During the tests execution this test data is changed. One of requirements for a good test is repeatedness. Test have to be launched any times and should complete with the same results.

So we need some sort of clean-up procedures for our tests in order to satisfy above requirement. Let’s consider what SpecFlow could offer us in this case.

Hooks

Hook is the statement of code that is executed after or before particular event. Here are the most useful from my mind:

  1. [AfterScenario(“MyTag”)] – code is executed after all the scenarios marked with @MyTag.
  2. [BeforeScenario(“MyTag”)] – code is executed before all the scenarios marked with @MyTag

You could find more info about SpecFlow hooks on the project wiki here.

Let’s consider very simple scenario, where we need some sort of clean-up procedures:

Scenario: User changes company
Given: User is work for a company ABC
When: User submits to work for the company DFE
Then: User is now working for the DFE

What we could expect when test is finished? That the user is now bound to the DFE company. But what happened when we try to repeat this test. Sure it will fail as now user is not bound to the ABC, but to DFE. What we need is to organise cleanup action after test execution, so user will be bound to the ABC again.

How could we achieve such behaviour. Well using the hooks in such way.

[AfterScenario]
public void AfterScenario()
{
    var title = ScenarioContext.Current.ScenarioInfo.Title;
    if (title == "User changes company")
    {
        //do some actions to rebound user to the initial company
    }
    if (title == "SomeOtherTest")
    {
        //do some clean-up for that
    }
}

Usually logic inside if statement might be quite difficult, as it requires some action to do and reuse the our Page Objects. So AfterScenario method become bloated and hard to read. This is not how we want to work.

This is when declarative programming comes in game. So let’s replace above statement with 2 separate methods, market with our custom attribute;

[ScenarioCleanup("User changes company")]
public void ResetBindedUser()
{
    //do some actions to rebound user to the initial company
}

[ScenarioCleanup("SomeOtherTest")]
public void CleanupAfterOtherTest()
{
    //do some other actions
}

Code for newly created attribute;

public class ScenarioCleanup : Attribute
{
    public string ScenarioTitle { get; set; }

    public ScenarioCleanup(string scenarioTitle)
    {
        ScenarioTitle = scenarioTitle;
    }
}

And finally the code where all parts are gathered;

[Binding]
public class TestBehaviour
{
    public void BeforeTestRun()
    {
        _teardownActions = BuildDictionary(this.GetType());
    }

    [AfterScenario()]
    public void AfterScenario()
    {
        _commonTestBehaviour.ResolveScenarioCleanup(ScenarioContext.Current.ScenarioInfo.Title, this);
        //some other code, like disposing selenium driver or so
    }

    [ScenarioCleanup("User changes company")]
    public void ResetBindedUser()
    {
        //do some actions to rebound user to the initial company
    }

    public void ResolveScenarioCleanup(string title, object instance)
    {
        MethodInfo cleanupMethod;
        if (_teardownActions.TryGetValue(title, out cleanupMethod))
        {
            cleanupMethod.Invoke(instance, null);
        }
    }

    private Dictionary<string, MethodInfo> _teardownActions;

    private Dictionary<string, MethodInfo> BuildDictionary(Type type)
    {
        var withAttrs = type.GetMethods()
            .Where(m => m.GetCustomAttributes(typeof(ScenarioCleanup), false).Length > 0)
            .ToDictionary(key => ((ScenarioCleanup[])key.GetCustomAttributes(typeof(ScenarioCleanup), false))[0].ScenarioTitle, val => val);
        return withAttrs;
    }
}

The code is pretty straightforward. Before TestRun via reflection we gather up all the cleanup methods inside. When scenario is executed, wi hit inside AfterScenario() method, where we look inside cleanup actions dictionary in order to know if we need to execute something. If yes, then call that method via reflection;

If we make such cleanups, what then should be inside the Given step? In my opinion in Given steps, there should be only Assertion about the data, not changing the data; All changes should inside When block;
In our scenario: Given: User is work for a company ABC
Inside a code bounded to this step, should be something like that;

[Given("User is work for a company (.*)")]
public void GivenUserWorksForCompany(string companyName)
{
    //navigate to user page
    var user = GetUser();
    if (!user.BoundTo(companyName))
    {
        throw new DataIntegrityException("User is not bound to the company " + companyName);
    }
}
Posted in Selenium, SpecFlow | Tagged , , | Leave a comment

ASP.NET WEB API documentation using Swagger – Global operation for 500 response code

Some api response codes, such as 500 for Internal server error, are the same for all API endpoints.
Let’s see how to implement such general behaviour using Swashbuckle.

Add below classes:

public class InternalServerErrorModel
{
    public string Message { get; set; }

    public string ExceptionMessage { get; set; }

    public string ExceptionType { get; set; }

    public string StackTrace { get; set; }
}

public class InternalServerErrorResponseFilter : IOperationFilter
{
    public void Apply(Operation operation, SchemaRegistry schemaRegistry, ApiDescription apiDescription)
    {
        var schema = schemaRegistry.GetOrRegister(typeof(InternalServerErrorModel));

        if (!operation.responses.ContainsKey("500"))
        {
            operation.responses.Add(new KeyValuePair<string, Response>("500", new Response()
            {
                description = "Internal server error occured",
                schema = schema
            }));
        }
    }
}

Register new operation filter in SwaggerConfig.cs

  c.OperationFilter<InternalServerErrorResponseFilter>(); 

And after running the application, each endpoint descriptions is decorated with:

Important!

Model above is default for asp.net web api, but it is security issue to expose internal details such Exception stack trace to outside world. Much better is to log exceptions to database using, for example, ELMAH library, and return just Error ID to clients.

Posted in Documentation, WEB API | Tagged , | Leave a comment

ASP.NET WEB API documentation using Swagger – Extend schema generation using SchemaFilters and FluentValidation rules

In this post we will see how to extend schema generation using FluentValidation rules.
We will see how to display min/max constraints for Integer type in Swagger UI and how to extend default examples to show valid email for email attribute.To get overall view about Schema filter, follow the official documentation at: Swashbuckle – Schema filters

Add FluentValidation to project + new API endpoint for demo

Install latest FluentValidation.WebAPI package into SwaggerDemo project.
Now let’s create another API endpoint, add new UsersController, model and validation:

public class UsersController : ApiController
    {
        public IHttpActionResult Post(UserModel model)
        {
            return Ok();//stub
        }
    }

    [Validator(typeof(UserModelValidator))]
    public class UserModel
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public string Email { get; set; }
        public int YearOfBirth { get; set; }
    }

    public class UserModelValidator : AbstractValidator
    {
        public UserModelValidator()
        {
            RuleFor(m => m.Email)
                .NotEmpty()
                .SetValidator(new EmailValidator());

            RuleFor(m => m.FirstName)
                .NotEmpty()
                .Length(3, 25);

            RuleFor(m => m.LastName)
                .Length(3, 25);

            RuleFor(m => m.YearOfBirth)
                .NotEmpty()
                .InclusiveBetween(1991, 2017);
        }
    }

Implement Schema updates based on validation rules

We will create and use IFluentValidatorSchemaUpdater interface and implement it for every suitable validator. With this design, we automatically conform to Single responsibility and Interface segregation principle for each implementation

    public interface IFluentValidatorSchemaUpdater
    {
        void Update(Schema schema, PropertyRule rule);
    }

    public abstract class FluentValidatorSchemaUpdater
        : IFluentValidatorSchemaUpdater where TValidator : IPropertyValidator
    {
        public void Update(Schema schema, PropertyRule rule)
        {
            var validator = rule.Validators.FirstOrDefault(v => v.GetType() == typeof(TValidator));

            if (validator != null)
            {
                Update(schema, rule, (TValidator)validator);
            }
        }

        protected abstract void Update(Schema schema, PropertyRule rule, TValidator validator);
    }

    public class RequriedFieldFluentValidatorSchemaUpdater
        : FluentValidatorSchemaUpdater
    {
        protected override void Update(
            Schema schema,
            PropertyRule rule,
            NotEmptyValidator validator)
        {
            if (schema.required == null)
            {
                schema.required = new List();
            }

            schema.required.Add(rule.PropertyName);
        }
    }

    public class EmailFluentValidatorSchemaUpdater
        : FluentValidatorSchemaUpdater
    {
        protected override void Update(
            Schema schema,
            PropertyRule rule,
            EmailValidator validator)
        {
            schema.properties[rule.PropertyName].example = "string.string@example.com";
        }
    }

    public class StringMinMaxFluentValidatorSchemaUpdater : FluentValidatorSchemaUpdater
    {
        protected override void Update(
            Schema schema,
            PropertyRule rule,
            LengthValidator validator)
        {
            schema.properties[rule.PropertyName].minLength = validator.Min;
            schema.properties[rule.PropertyName].maxLength = validator.Max;
        }
    }

    public class IntMinMaxFluentValidatorSchemaUpdater : FluentValidatorSchemaUpdater
    {
        protected override void Update(
            Schema schema,
            PropertyRule rule,
            InclusiveBetweenValidator validator)
        {
            if (rule.Expression.ReturnType == typeof(int))
            {
                schema.properties[rule.PropertyName].example = (int)validator.From;
                schema.properties[rule.PropertyName].minimum = (int)validator.From;
                schema.properties[rule.PropertyName].maximum = (int)validator.To;
            }
        }
    }

    public class FormatFluentValidatorSchemaUpdater
        : FluentValidatorSchemaUpdater
    {
        protected override void Update(
            Schema schema,
            PropertyRule rule,
            RegularExpressionValidator validator)
        {
            schema.properties[rule.PropertyName].format = validator.Expression;
        }
    }

Register Schema filter to Swagger configuration

Now add new schema filter, here we manually inject all the IFluentValidatorSchemaUpdater implementations, but it could also be done using dependency injection library, such as Ninject.

The implementation is pretty straightforward:

  • For each class decorated with ValidatorAttribute (from FluentValidator namespace), such as UserModel,
  • Take properties with validation rules,
  • Update Swagger schema according to registered schema updaters
 public class FluentValidationSchemaFilter : ISchemaFilter
    {
        public void Apply(Schema schema, SchemaRegistry schemaRegistry, Type type)
        {
            var updaters = new List{
                new StringMinMaxFluentValidatorSchemaUpdater(),
                new FormatFluentValidatorSchemaUpdater(),
                new RequriedFieldFluentValidatorSchemaUpdater(),
                new EmailFluentValidatorSchemaUpdater(),
                new IntMinMaxFluentValidatorSchemaUpdater()
            };

            var customAttribute = type.GetCustomAttribute();

            if (customAttribute == null)
            {
                return;
            }

            var validator = (IValidator)Activator.CreateInstance(customAttribute.ValidatorType);
            var descriptor = validator.CreateDescriptor();
            var membersWithValidations = descriptor.GetMembersWithValidators();

            foreach (var member in membersWithValidations)
            {
                var rules = descriptor.GetRulesForMember(member.Key);

                foreach (var r in rules)
                {
                    updaters.ForEach(u => u.Update(schema, (PropertyRule)r));
                }
            }
        }
    }

The last step is to register SchemaFilter in SwaggerConfig

 c.SchemaFilter();

Now after running project we will see below improvements:

  • Email is displayed not as autogenerated string, but as valid email address
  • YearOfBirth is generated as valid integer based on declared validation rules
  • When hover over the Model field, such as YearOfBirth, in pop-up it is possible to see minimum and maximum values. The same works for strings as well (min/max length)

Summary:

We have seen how to extend Swagger generated schema using FluentValidation rules.
It is a quite common technique to mark particular property/method with custom attribute, that could be used to auto-generate documentation.

Posted in ASP.NET, Documentation, WEB API | Tagged , | Leave a comment

ASP.NET Web API documentation using Swagger – Useful links

Some of good additions to Swagger might be quite difficult to find so I put useful links here:

Generating model examples:

Using markdown in descriptions:

It is possible to render really cool description in Swagger UI using Github markdown, for example nice table:

Open API specification:

Posted in Documentation, WEB API | Tagged , | Leave a comment

ASP.NET Web API documentation using Swagger – Use schema mapper for generic data types

If get back to previous post, you could see that descriptions for StartDate and EndDate are duplicated.
DateTime description shall be consistent across all API, so to not repeat yourself let’s use MapType feature of Swashbuckle configuration.

Open SwaggerConfig and find section with c.MapType (it is commented by default). Now let’s map DateTime to the specific schema:

c.MapType<System.DateTime>(() => new Swashbuckle.Swagger.Schema
{
    type = "string",
    format = "date-time",
    description = 
        @"Date-time string in <a href=""https://en.wikipedia.org/wiki/ISO_8601#UTC\"">ISO 8601 format</a>.
          Date-time without time-zone is treated as UTC."
});

Now, we could remove explicit descriptions from DiscountModel:

[Required]
public DateTime StartDate { get; set; }

[Required]
public DateTime EndDate { get; set; }

If now open Swagger page, you will see consistent descriptions for DateTime field:

One more thing is that you could override the description by using XML comments over the model property:

/// 
<summary>
/// Overriden description
/// </summary>

[Required]
public DateTime StartDate { get; set; }

Now Swagger-UI page looks like this:

Posted in Documentation, WEB API | Tagged , | Leave a comment

ASP.NET Web API documentation using Swagger – Add custom headers

In this post, we will look into how to add custom headers to your Swagger documentation using Swashbuckle.
We will add well-known “Accept-Language” header with conditionally visibility based on API method!

Setup application culture based on Accept-Language header

First, add CultureHandler and enable it into WEB API pipeline (for better understanding on how ASP.NET WEB API pipeline works and what are delegating handler, please refer to the-asp-net-web-api-2-http-message-lifecycle-in-43-easy-steps-2/

public class CultureHandler : DelegatingHandler
{
    private static readonly string[] _acceptedCultures = { "en-GB", "de-DE" };
    private static readonly string _defaultCulture = "en-GB";

    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        Thread.CurrentThread.CurrentCulture = new CultureInfo(_defaultCulture);
            
        if (request.Headers.AcceptLanguage != null)
        {
            foreach (var v in request.Headers.AcceptLanguage.OrderBy(al => al.Quality))
            {
                if (_acceptedCultures.Contains(v.Value))
                {
                    Thread.CurrentThread.CurrentCulture = new CultureInfo(v.Value);
                }
            }
        }

        return base.SendAsync(request, cancellationToken);
    }
}

Next, register CultureHandler in WebApiConfig:

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        config.MapHttpAttributeRoutes();

        config.MessageHandlers.Add(new CultureHandler());

        config.Routes.MapHttpRoute(
            name: "DefaultApi",
            routeTemplate: "api/{controller}/{id}",
            defaults: new { id = RouteParameter.Optional }
        );
    }
}

Display Accept-Language header in Swagger UI

Ok, now it is time to add the possibility to display new header in Swagger UI.
For that, we will extend Swashbuckle behavior by introducing CultureAwareOperationFilter:

public class CultureAwareOperationFilter : IOperationFilter
{
    public void Apply(Operation operation, SchemaRegistry schemaRegistry, ApiDescription apiDescription)
    {
        if (operation.parameters == null)
        {
            operation.parameters = new List<Parameter>();
        }

        operation.parameters.Add(new Parameter
        {
            name = "Accept-Language",
            @in = "header",
            type = "string",
            required = true,
            @enum = new [] { "en-GB", "de-DE"}
        });
    }
}

Register new Operation filter in SwaggerConfig

c.OperationFilter<CultureAwareOperationFilter>();

To test new behavior let’s add new controller with simple localized content

public class ProductController : ApiController
{
    [Route("products")]
    public IHttpActionResult Get()
    {
        if (Thread.CurrentThread.CurrentCulture.Name == "de-DE")
        {
            return Ok("I am German product!");
        }

        return Ok("I am English product!");
    }
}

Now, it is time to run the project, if go by URI: http://localhost:50217/swagger/ui/index#!/Product/Product_Get, you could now see Accept-Language dropdown with English and German culture inside:

Now if select de-DE there, as a response you will get “I am German product!”.

Restricting Accept-Language header for specific APIs

If you now look at Discount API, you will see that it also got new header selection. That is quite misleading as API does not provide localization. Let’s try to fix this by extending CultureAwareOperationFilter. To make OperationFilter configuration decoupled from actual actions we will be using CultureAwareAttribute that will decorate specific methods.

public class CultureAwareAttribute : Attribute
{
}

Extend CultureAwareOperationFilter

public void Apply(Operation operation, SchemaRegistry schemaRegistry, ApiDescription apiDescription)
{
    if (!apiDescription.ActionDescriptor.GetCustomAttributes<CultureAwareAttribute>().Any())
    {
        return;
    }

    ...
}

The last step is to decorate required actions with CultureAwareAttribute,

[Route("products")]
[CultureAware]
public IHttpActionResult Get()

Now if we open Swagger, we will see language header selection only for Products API, but it won’t appear for others.

Another usage example of operation filter

The operation is displayed as dropdown if @enum property is specified.

Another alternative is to specify @defaultValue


operation.parameters.Add(new Parameter
{
name = "X-Custom-Header",
@in = "header",
type = "string",
required = true,
@default = "Default value"
});

In that case operation is displayed as textbox. That representation is often used to supply security key for API usage.

Summary:

We have looked into how to use operation filters to display custom headers in Swagger UI.
For basic usage, remember that @enum option is rendered as dropdown of choices, and as textbox otherwise.

Posted in Documentation, WEB API | Tagged , | Leave a comment

ASP.NET Web API documentation using Swagger – Introduction

Providing always up-to-date and exhaustive documentation for your WEB API could be quite a challenging task.
Swagger and Swashbuckle did a great job to provide uniform documentation for any kind of API.
Let’s look into this using simple demo discount API example.

Setup ASP.NET WEB API project

First, create empty ASP.NET Web API project.
Add new controller – DiscountController inside Controllers folder.

using System;
using System.Web.Http;
using SwaggerDemo.Models;

namespace SwaggerDemo.Controllers
{
    public class DiscountController : ApiController
    {
        [Route("discounts")]
        public IHttpActionResult Post(DiscountModel model)
        {
            model.Id = 2017;
            return Created("", model);
        }

        [Route("discounts/{id}")]
        public IHttpActionResult Get(int id)
        {
            return Ok(new DiscountModel
            {
                Id = id,
                StartDate = new DateTime(2017, 10, 31),
                EndDate = new DateTime(2019, 10, 31),
                Type = "PercentOff",
                Value = 35
            });
        }
    }
}


DiscountModel class:

using System;

namespace SwaggerDemo.Models
{
    public class DiscountModel
    {
        public int Id { get; set; }
        public DateTime StartDate { get; set; }
        public DateTime EndDate { get; set; }
        public string Type { get; set; }
        public decimal Value { get; set; }
    }
}

Now if we run F5 or Debug in Visual Studio, our new API is accessible at http://localhost:50217/discounts/25 (50217 is port number, could be different on your machine)

Add Swashbuckle to the project

Swashbukle page says: Seamlessly adds a swagger to WebApi projects! And it works in that way, all you need is to install it from Nuget and you are ready to start.
Now rerun application, you could now access SwaggerUI page by http://localhost:50217/swagger/ui/index

Add more documentation

Let’s look into what Swashbuckle generated by default. First look at model:

Questions that immediately appear if want to start using API:

  1. What is correct format for dates?
  2. What are possible options for field ‘Type’
  3. How much decimal point is supported for  Value field?
  4. What fields are really optional?

Let’s try to answer all those questions by providing XML comments for model properties.

    public class DiscountModel
    {
        /// <summary>
        /// Discount unique Id, auto-generated
        /// </summary>
        public int Id { get; set; }

        /// <summary>
        /// Date-time string in <a href="https://en.wikipedia.org/wiki/ISO_8601#UTC">ISO 8601 format</a>.
        /// Date-time without time-zone is treated as UTC.
        /// </summary>
        [Required]
        public DateTime StartDate { get; set; }

        /// <summary>
        /// Date-time string in <a href="https://en.wikipedia.org/wiki/ISO_8601#UTC">ISO 8601 format</a>.
        /// Date-time without time-zone is treated as UTC.
        /// </summary>
        [Required]
        public DateTime EndDate { get; set; }

        /// <summary>
        /// Possible values are: PercentOff and MoneyOff
        /// </summary>
        [Required]
        public string Type { get; set; }

        /// <summary>
        /// Max number of decimal places is 2. E.g. provided 35.355 will be truncated to 35.35.
        /// </summary>
        [Required]
        public decimal Value { get; set; }
    }
  1. Enable XML comments in SwaggerConfig
    Uncomment and edit line:

    c.IncludeXmlComments(System.Web.Hosting.HostingEnvironment.MapPath("~/bin/SwaggerDemo.xml")); 
  2. Enable XML documentation output during project build
    Go Project Properties => Build section => Check XML documentation file checkbox.

Let’s build and run the project, now model looks much nicer providing guidance on how to use discount model.

Summary:

Enabled Swagger documentation for Web API project.
Added model property XML comments to provide information about expected input options and its format.
Used DataAnnotations to document required model properties.

Posted in Documentation, WEB API | Tagged , | Leave a comment

AutoFixture > Using [Frozen] attribute on test method parameter

[Frozen] attribute is used to freeze instances of objects. It is applied on test method parameters.
We will consider simple examples with string comparison test.

Without [Frozen] attribute

using Ploeh.AutoFixture.Xunit2;
using Xunit;

[Theory, InlineAutoData]
public void No_Frozen_Attribute_Generates_Random_Strings(
	string one,
	string two)
{
	Assert.Equal(one, two);
}

Result: Failed

Using [Frozen] attribute

Applying [Frozen] on the first parameter of type string, will return the same values for all subsequent parameters.

[Theory, InlineAutoData]
public void Frozen_On_Parameter_Name_Freeze_Object_Instance(
   [Frozen]string one,
	string two)
{
	Assert.Equal(one, two);
}

Result: Passed

Using [Frozen] attribute – Order is important!

Applying [Frozen] attribute on the second parameter will generate two random strings.
But if we have third parameter of type ‘string’, then it will have the same value as ‘two’.

[Theory, InlineAutoData]
public void Frozen_On_Parameter_Name_Order_Is_Important(
	string one,
	[Frozen]string two)
{
	Assert.Equal(one, two);
}

Result: Failed

Posted in Unit test | Tagged , | Leave a comment