The Evolution of .NET Configuration

Every programmer imagined (well, or might want to imagine) himself/herself as a pilot in the situation when you have a huge project, a huge panel of sensors, metrics and switches using which you can easily configure everything. Well, at least you do not have to run and manually lift the chassis. Both metrics and graphs are a good thing, but today I want to tell you about those tumblers and buttons that can change the parameters of an aircraft behavior and configure it.

Everyone knows that configuration is a very important part of programming. Different specialists use different approaches to configure their applications, and we believe that there is nothing complicated about it, but is it really that simple? Let’s look at the ‘before’ and ‘after’ of the process of configuration and find out about the details: the work of different items, new features and how to use them. Those who are not familiar with configuring in .NET Core will get the basics, and those who are already familiar with it will get food for thought and will be able to use new approaches in their daily work.

Before .NET Core Configuration

In 2002, the .NET Framework was introduced, and since it was the time of the XML hype, the developers from Microsoft decided to use it everywhere, and as a result, we have got XML configurations that are still alive. At the head of the table, we have a static class ConfigurationManager using which we get string representations of settings. The configuration itself looked sort of like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="Title" value=".NET Configuration evo" />
    <add key="MaxPage" value="10" />
  </appSettings>
</configuration>

The issue has been solved, developers have got a setting option, which is better than INI files, but having its own peculiarities. So, for example, support for different setting values of different types of application environments is implemented using XSLT transformations of the configuration file. We can define our own XML schemas for elements and attributes if we want more complex data grouping. Key-value pairs are of a string type, but if you need a number or a date, then use your imagination:

string title = ConfigurationManager.AppSettings["Title"];
int maxPage = int.Parse(ConfigurationManager.AppSettings["MaxPage"]);

In 2005 configuration sections were added. They allowed grouping parameters, building your own schemes, avoiding naming conflicts. Also *.settings files and a special designer for them were presented.

*.settings file designer

We have got a possibility to have generated, strongly typed class that represents configuration data. The designer allows you to conveniently edit the values, sorting by editor columns is available. The data is retrieved using the Default property of the generated class, which provides a Singleton configuration object.

DateTime date = Properties.Settings.Default.CustomDate;
int displayItems = Properties.Settings.Default.MaxDisplayItems;
string name = Properties.Settings.Default.ApplicationName;

The application fields of configuration parameter values were also added. The User field is responsible for user data, which can be changed by the user and saved during program execution. Data is saved in a separate file along the path %AppData%\*Application name *. The Application field allows you to retrieve parameter values without the possibility of user redefinition.

Despite the good intentions, everything became more complicated.

  • In fact, these are the same XML files that can grow in size faster and, as a result, are inconvenient to read.
  • The configuration is read from the XML file once, and you need to reboot the application to change the configuration data.
  • Classes generated from *.settings files are marked with the sealed modifier, so this class can’t be inherited. In addition, this file can be changed, but if any regeneration takes place, you lose everything you have written yourself.
  • Working with data only on a key-value basis. To get a structured approach to working with configurations, you need to additionally implement this yourself.
  • The data source can only be a file, external providers are not supported.
  • Plus, we have a human factor. Private parameters can get into the version control system and become disclosed.

All of these issues remain in the .NET Framework to this day.

Configuration in .NET Core

In .NET Core, configuration was redesigned and everything was made from the scratch, the static class ConfigurationManager was removed and many of the ‘before’ issues were solved. What have we got new? As before, we have the stage of forming the configuration data and the stage of consuming this data but with a more flexible and extended life cycle.

Customization and adding configuration data

We can use many sources for the stage of data generation, not limiting ourselves only to files. Configuration customization is done through IConfgurationBuilder. This is the basis to which we can add data sources. NuGet packages are available for various types of sources:

Format Extension method to add a source to IConfigurationBuilder NuGet package
JSON AddJsonFile Microsoft.Extensions.Configuration.Json
XML AddXmlFile Microsoft.Extensions.Configuration.Xml
INI AddIniFile Microsoft.Extensions.Configuration.Ini
Command line arguments AddCommandLine Microsoft.Extensions.Configuration.CommandLine
Environment variables AddEnvironmentVariables Microsoft.Extensions.Configuration.EnvironmentVariables
User secrets AddUserSecrets Microsoft.Extensions.Configuration.UserSecrets
KeyPerFile AddKeyPerFile Microsoft.Extensions.Configuration.KeyPerFile
Azure KeyVault AddAzureKeyVault Microsoft.Extensions.Configuration.AzureKeyVault

Each source is added as a new layer and redefines the parameters with matching keys. Here is the Program.cs example that comes by default in the ASP.NET Core Application Template (version 3.1).

public static IHostBuilder CreateHostBuilder(string[] args) => 
    Host.CreateDefaultBuilder(args).ConfigureWebHostDefaults(webBuilder => 
        { webBuilder.UseStartup<Startup>(); });

I want to focus on CreateDefaultBuilder. Inside the method, we will look at the initial configuration of sources.

public static IWebHostBuilder CreateDefaultBuilder(string[] args)
{
    var builder = new WebHostBuilder();
    ...
    builder.ConfigureAppConfiguration((hostingContext, config) =>
    {
        IHostingEnvironment env = hostingContext.HostingEnvironment;
        config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
              .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);
        if (env.IsDevelopment())
        {
            Assembly appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
            if (appAssembly != null)
            {
                config.AddUserSecrets(appAssembly, optional: true);
            }
        }
        config.AddEnvironmentVariables();
        if (args != null)
        {
            config.AddCommandLine(args);
        }
    })
            
    ...
    return builder;
}

We see that the base for the entire configuration is the appsettings.json file. Further, if there is a file for a specific environment, it will have a higher priority, and thereby redefine the matching values of the base file. The same is done with each subsequent source. The order of addition affects the final value. Visually, it looks like this:

The scheme of creating .NET configuration

If you want to use your own order, you can simply clear it and define in a way you need.

Host.CreateDefaultBuilder(args)
    .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); })
    .ConfigureAppConfiguration((context,
                                builder) =>
     {
         builder.Sources.Clear();
         
         //Custom source order
     });

Each configuration source consists of two parts:

  • Implementation of IConfigurationSource. Provides a source of configuration values.
  • Implementation of IConfigurationProvider. Converts the original data to a resulting key-value.

By implementing these components, we can get our own data source for configuration. Here is an example of how you can get parameters from a database through the Entity Framework.

How to use and retrieve data

Now the way we can customize and add data is clear, let’s take a look at how we can use this data and how to get it more conveniently. The new approach to configuring projects makes a big bias towards the popular JSON format. This is not surprising, because using it we can build any data structures, group data and have a readable file at the same time. The following configuration file is an example:

{
  "Features" : {
    "Dashboard" : {
      "Title" : "Default dashboard",
      "EnableCurrencyRates" : true
    },
    "Monitoring" : {
      "EnableRPSLog" : false,
      "EnableStorageStatistic" : true,
      "StartTime": "09:00"
    }
  }
}

All data forms a flat key-value dictionary, the configuration key is formed from the entire file key hierarchy for each value. A similar structure would have the following data set:

Features:Dashboard:Title Default dashboard
Features:Dashboard:EnableCurrencyRates true
Features:Monitoring:EnableRPSLog false
Features:Monitoring:EnableStorageStatistic true
Features:Monitoring:StartTime 09:00

We can get the value using the IСonfiguration object. Here’s the way we can get the parameters:

string title = Configuration["Features:Dashboard:Title"];
string title1 = Configuration.GetValue<string>("Features:Dashboard:Title");
bool currencyRates = Configuration.GetValue<bool>("Features:Dashboard:EnableCurrencyRates");
bool enableRPSLog = Configuration.GetValue<bool>("Features:Monitoring:EnableRPSLog");
bool enableStorageStatistic = Configuration.GetValue<bool>("Features:Monitoring:EnableStorageStatistic");
TimeSpan startTime = Configuration.GetValue<TimeSpan>("Features:Monitoring:StartTime");

And this is quite good, because we have a convenient way to get data of a required data type, but it is not as cool as we would like it to be. If we receive data as in the above example, we will end up with a repetitive code and make mistakes in the names of the keys. Instead of individual values, you can have a complete configuration object. Binding data to an object using the Bind method will help us. Example of class and data retrieval:

public class MonitoringConfig
{
    public bool EnableRPSLog { get; set; }
    public bool EnableStorageStatistic { get; set; }
    public TimeSpan StartTime { get; set; }
}
var monitorConfiguration = new MonitoringConfig();
Configuration.Bind("Features:Monitoring", monitorConfiguration);
var monitorConfiguration1 = new MonitoringConfig();
IConfigurationSection configurationSection = Configuration.GetSection("Features:Monitoring");
configurationSection.Bind(monitorConfiguration1);

In the first case, we bind by the section name, and in the second, we get a section and bind from it. The section allows you to work with a partial view of the configuration. In this way you can control the data set you are working with. Sections are also used in standard extension methods, for example, the ‘ConnectionStrings’ section is used to get a connection string.

string connectionString = Configuration.GetConnectionString("Default");
public static string GetConnectionString(this IConfiguration configuration, string name)
{
    return configuration?.GetSection("ConnectionStrings")?[name];
}

Options—typed configuration view

Creating a configuration object manually and binding it to data is unpractical, but the solution is to use Options. Options are used to get a typed view of a configuration. The view class must be public with a constructor without parameters and public properties for assigning a value, the object is filled through reflection. More details can be found in the source.

To start using Options, we need to register the configuration type with the help of the Configure extension method for IServiceCollection indicating the section that we will project onto our class.

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers();
    services.Configure<MonitoringConfig>(Configuration.GetSection("Features:Monitoring"));
}

After that, we can receive configurations by injection a dependency to the IOptions, IOptionsMonitor, IOptionsSnapshot interfaces. We can get the MonitoringConfig object from the IOptions interface with the help of the Value property.

public class ExampleService
{
    private IOptions<MonitoringConfig> _configuration;
    public ExampleService(IOptions<MonitoringConfig> configuration)
    {
        _configuration = configuration;
    }
    public void Run()
    {
        TimeSpan timeSpan = _configuration.Value.StartTime; // 09:00
    }
}

A feature of the IOptions interface is that in the dependency injection container, the configuration is registered as an object with the Singleton life cycle. The first time a value is requested by the Value property, an object is initialized with data that exists as long as this object exists. IOptions does not support data update. There are IOptionsSnapshot and IOptionsMonitor interfaces to support updates.

The IOptionsSnapshot in the dependency injection container is registered with the Scoped life cycle, which makes it possible to get a new configuration object for a request with a new container field. For example, during one web request we will receive the same object, but we will receive a new object with updated data for a new request.

IOptionsMonitor is registered as a Singleton, with the only difference that each configuration is received with the actual data at the time of the request. In addition, IOptionsMonitor allows you to register a configuration change event handler if you need to respond to the data change event itself.

public class ExampleService
{
    private IOptionsMonitor<MonitoringConfig> _configuration;
    public ExampleService(IOptionsMonitor<MonitoringConfig> configuration)
    {
        _configuration = configuration;
        configuration.OnChange(config =>
        {
            Console.WriteLine("Configuration has changed");
        });
    }
    
    public void Run()
    {
        TimeSpan timeSpan = _configuration.CurrentValue.StartTime; // 09:00
    }
}

It is also possible to get IOptionsSnapshot and IOptionsMonitor by name. This is necessary if you have several configuration sections corresponding to one class, and you want to get a specific one. For example, we have the following data:

{
  "Cache": {
    "Main": {
      "Type": "global",
      "Interval": "10:00"
    },
    "Partial": {
      "Type": "personal",
      "Interval": "01:30"
    }
  }
}

The type to be used for the projection:

public class CachePolicy
{
    public string Type { get; set; }
    public TimeSpan Interval { get; set; }
}

We register configurations with a specific name:

services.Configure<CachePolicy>("Main", Configuration.GetSection("Cache:Main"));
services.Configure<CachePolicy>("Partial", Configuration.GetSection("Cache:Partial"));

We can receive values as follows:

public class ExampleService
{
    public ExampleService(IOptionsSnapshot<CachePolicy> configuration)
    {
        CachePolicy main = configuration.Get("Main");
        TimeSpan mainInterval = main.Interval; // 10:00
            
        CachePolicy partial = configuration.Get("Partial");
        TimeSpan partialInterval = partial.Interval; // 01:30
    }
}

If you look at the source of the extension method with the help of which we register the configuration type, you can see that the default name is Options.Default, which is an empty string. So implicitly we always transfer the name for the configurations.

public static IServiceCollection Configure<TOptions>(this IServiceCollection services, IConfiguration config) where TOptions : class
            => services.Configure<TOptions>(Options.Options.DefaultName, config);

Since a configuration can be represented by a class, we can also add parameters value validation by marking up properties using validation attributes from the System.ComponentModel.DataAnnotations namespace. For example, we specify that the value for the Type property must be mandatory. But we also need to indicate that validation should take place when registering the configuration. An extension method ValidateDataAnnotations is used to do so.

public class CachePolicy
{
    [Required]
    public string Type { get; set; }
    public TimeSpan Interval { get; set; }
}
services.AddOptions<CachePolicy>()
        .Bind(Configuration.GetSection("Cache:Main"))
        .ValidateDataAnnotations();

The peculiarity of such a validation is that it will happen only at the moment of receiving the configuration object. This makes it difficult to understand that the configuration is not valid when the application starts. There is a task on GitHub to solve that problem. One of the solutions to this problem can be the approach presented in the article Adding validation to strongly typed configuration objects in ASP.NET Core.

Disadvantages of Options and how to escape them

Configuring via Options also has its disadvantages. We need to add a dependency to use it, and we need to access the Value/CurrentValue property each time to get a value object. You can get a purer code by getting a pure configuration object without the Options wrapper. The simplest solution to the problem may be additional registration in the container of a pure configuration type dependency.

services.Configure<MonitoringConfig>(Configuration.GetSection("Features:Monitoring"));
services.AddScoped<MonitoringConfig>(provider => provider.GetRequiredService<IOptionsSnapshot<MonitoringConfig>>().Value);

The solution is straightforward, we do not force the final code to know about IOptions, but we lose the flexibility for additional configuration actions if we need them. To solve this problem, we can use the ‘Bridge’ pattern, which will allow us to get an additional layer where we can perform additional actions before receiving an object.

To achieve the goal, we need to refactor the current example code. Since the configuration class has a restriction in the form of a constructor without parameters, we can’t transfer the IOptions/IOptionsSnapshot/ IOptionsMontitor object to the constructor; we need to separate the configuration reading from the final view to transfer the object.

For example, let’s say that we want to specify the StartTime property of the MonitoringConfig class with a string representation of minutes with a value of ‘09’, which doesn’t fit the standard format.

public class MonitoringConfigReader
{
    public bool EnableRPSLog { get; set; }
    public bool EnableStorageStatistic { get; set; }
    public string StartTime { get; set; }
}
public interface IMonitoringConfig
{
    bool EnableRPSLog { get; }
    bool EnableStorageStatistic { get; }
    TimeSpan StartTime { get; }
}
public class MonitoringConfig : IMonitoringConfig
{
    public MonitoringConfig(IOptionsMonitor<MonitoringConfigReader> option)
    {
        MonitoringConfigReader reader = option.Value;
        
        EnableRPSLog = reader.EnableRPSLog;
        EnableStorageStatistic = reader.EnableStorageStatistic;
        StartTime = GetTimeSpanValue(reader.StartTime);
    }
    
    public bool EnableRPSLog { get; }
    public bool EnableStorageStatistic { get; }
    public TimeSpan StartTime { get; }
    
    private static TimeSpan GetTimeSpanValue(string value) => TimeSpan.ParseExact(value, "mm", CultureInfo.InvariantCulture);
}

To get a pure configuration, we need to register it in the dependency injection container.

services.Configure<MonitoringConfigReader>(Configuration.GetSection("Features:Monitoring"));
services.AddTransient<IMonitoringConfig, MonitoringConfig>();

This approach allows you to create a completely separate life cycle for the formation of a configuration object. It is possible to add your own data validation, or additionally implement a data decryption stage if you receive it in an encrypted form.

Ensuring data security

An important configuration task is data security. File configurations are insecure as the data is stored in clear text which is easy to read; the files are often in the same directory as the application. By mistake, you can commit the values to the version control system, which can disclose the data, but what if this is a public code! The situation is so common that there is a ready-made tool for finding such leaks—Gitleaks. There is a separate article, where you can find statistics and the variety of disclosed data.

Often a project must have separate parameters for different environments (Release/Debug, etc.). For example, as one of the solutions, you can use substitution of final values using the tools of continuous integration and delivery, but this option does not protect the data at the design stage. The User Secrets tool is designed to protect the developer. It is included in the .NET Core SDK package (3.0.100 and higher). What is the main principle of this tool? First, we need to initialize our project with the init command.

dotnet user-secrets init

The command adds a UserSecretsId element to the .csproj project file. With this parameter, we get a private storage that will store a regular JSON file. The difference is that it is not located in your project directory, so it will only be available on the current computer. The path for Windows is %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json, and for Linux and MacOS ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json. We can add the value from the example above with the set command:

dotnet user-secrets set "Features:Monitoring:StartTime" "09:00"

A complete list of available commands can be found in the documentation.

Data security in production is best ensured using specialized storage, such as: AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, Consul, ZooKeeper. There are ready-made NuGet packages to implement some storages, and in some cases, it is easy to implement storages yourself, since there is access to the REST API.

Conclusion

Modern issues require modern solutions. Along with the move away from monoliths to dynamic infrastructures, configuration approaches have also undergone changes. There is a need to be independent of the location and type of sources of configuration data, a need for a prompt response to data changes. .NET Core gave us a good tool for implementing all kinds of application configuration scenarios.

You Might Also Like

Blog Posts Testing Strategy in a Short-Term Project
May 12, 2021
The strategies and testing tactics used for long-term projects are not very suitable for small ones. Here we show, how to design a testing strategy for a short-term project and put it into practice.
Blog Posts Action Filters to Create Cleaner Code
January 25, 2021
There are many ways to solve the annoying problem of duplicate code. In this post, we show how action filters can be used to clean up the code.
Blog Posts Understanding Workflow: Why It Matters and How to Change It
December 08, 2020
In this article, we will analyse a few simple examples that demonstrate how a workflow may look, what its common practices are, when it is needed and how to change it.
1
8