2017

Entity Framework Core with SQLite Migration Limitations

This is part of what has turned into a series on Entity Framework Core with SQLite. The other parts can be found below.

Entity Framework Core with SQLite
Entity Framework Core Errors Using Add-Migration
Entity Framework Core with SQLite Scaffolding

The starting point of the code for this post can be found here.

Migration Limitations when using SQLite

SQLite’s ALTER TABLE is limited which in turn limits what Entity Framework Core can do via a migration. The official docs on the subject can be found here. These limitations are on the Entity Framework Team’s list of issues as an open enhancement and can be tracked here.

As long as you are just adding new tables or columns you would never notice the limitation, but if you have spelling problems like I do then the need to rename a column can be important. Thankfully things like ReSpeller (link is to the pro page, but a free version is available in ReSharpers extension manager) help with my spelling issues.

Unsupported example with a column rename

As an example of how to handle a migration that isn’t supported, we are going to rename the State property of the Contact class to Subregion.

Rename property on the model

Open the Contact class which can be found in the Models directory and make the following change.

Before:
public string State { get; set; }

After:
public string Subregion { get; set; }
Add a migration

With the property name change using the following command in the Package Manager Console to create a new migration.

Add-Migration RenameContactStateToSubregion -c ContactsDbContext

Which produces the following migration class.

public partial class RenameContactStateToSubregion : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.RenameColumn(
            name: "State",
            table: "Contacts",
            newName: "Subregion");
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.RenameColumn(
            name: "Subregion",
            table: "Contacts",
            newName: "State");
    }
}
Error trying to apply the migration

As expected when an attempt to apply the above migration results in the following exception.

System.NotSupportedException: SQLite does not support this migration operation (‘RenameColumnOperation’). For more information, see http://go.microsoft.com/fwlink/?LinkId=723262.

Modify migration to manually rename the column

Searching for how to rename a column in SQLite will turn up a lot of results including this from the official docs and answers like this on StackOverflow. The gist of the how to do a rename is to create a new table with the desired schema, copy the data from the original table, drop the old table, and finally rename the new table to match the original name.

Now knowing the process the migration above can be modified to apply SQL directly instead of using Entity Framework Core to generate the SQL. This can be done by using the Sql function of the MigrationBuilder class. The following is the resulting migration.

public partial class RenameContactStateToSubregion : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.Sql(
            @"PRAGMA foreign_keys = 0;

              CREATE TABLE Contacts_temp AS SELECT *
                                            FROM Contacts;
              
              DROP TABLE Contacts;
              
              CREATE TABLE Contacts (
                  Id         INTEGER NOT NULL
                                     CONSTRAINT PK_Contacts PRIMARY KEY AUTOINCREMENT,
                  Address    TEXT,
                  City       TEXT,
                  Email      TEXT,
                  Name       TEXT,
                  Phone      TEXT,
                  PostalCode TEXT,
                  Subregion  TEXT
              );
              
              INSERT INTO Contacts 
              (
                  Id,
                  Address,
                  City,
                  Email,
                  Name,
                  Phone,
                  PostalCode,
                  Subregion
              )
              SELECT Id,
                     Address,
                     City,
                     Email,
                     Name,
                     Phone,
                     PostalCode,
                     State
              FROM Contacts_temp;
              
              DROP TABLE Contacts_temp;
              
              PRAGMA foreign_keys = 1;");
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
    }
}

You will notice that I didn’t bother doing the Down function, but the same idea would apply when trying to undo a migration. SQLiteStudio or similar tools can be used to generate the SQL above if SQL isn’t something you want to deal with.

Fix other references to the renamed field

This isn’t really the topic of this post, but I wanted to throw in a reminder that after a rename like this there are places that will need to be updated that the tooling may not have picked up. For example, make sure all your views are using the new column as well as any bind statements in your controllers.

Wrapping up

The first time I hit the need to rename a column and it resulted in an exception it was extremely frustrating. Over time as I learned what the tooling around SQLite provides it has become less of an issue. I look forward to seeing what the Entity Framework team does in the future around this issue. The finished code can be found here.

Entity Framework Core with SQLite Migration Limitations Read More »

Entity Framework Core with SQLite Scaffolding

This is the third in what is turning into a series of post about using SQLite with Entity Framework Core. This post is going to cover adding a migration, scaffolding a controller and related views, and a few things that are harder to do using SQLite. The following are the first two post.

Entity Framework Core with SQLite
Entity Framework Core Errors Using Add-Migration

Adding Model, DbContext, Controller, and Views

If you have any experience with Entity Framework Core or have read any of my past entries on the subject this section is going to repeat some of the same information, but I am including it so someone who is looking for a full example will have it.

Model

In the Models folder add a Contact class similar the following.

public class Contact
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Address { get; set; }
    public string City { get; set; }
    public string State { get; set; }
    public string PostalCode { get; set; }
    public string Phone { get; set; }
    public string Email { get; set; }
}
DbContext

In the Data folder add a ContactsDbContext that inherits from DbContext. The following is an example that auto applies migrations to a database, if you don’t need that functionality it can be dropped out.

public sealed class ContactsDbContext : DbContext
{
    private static bool _created;

    public DbSet<Contact> Contacts { get; set; }

    public ContactsDbContext(DbContextOptions<ContactsDbContext> options)
        : base(options)
    {
        if (_created) return;
        Database.Migrate();
        _created = true;
    }
}

Now that the application has a model and a related DbContext the following can be used to add a migration that will create a Contacts in the SQLite database. Run from the Package Manager console.

Add-Migration AddContacts -Context ContactsDbContext

Add-Migration is a Powershell command to add a migration (surprise!), AddContacts is the name of the migration and -Context ContactsDbContext is an argument that lets the command know which DbConext to use. The Context is only needed if your application has more than one DbContext.

Controller and Views

With the above complete Visual Studio provides some tooling that makes it very fast to create a controller with views for listing, adding, editing, and deleting items. To begin right-click on the Controllers folder and select Add > New Scaffolded Item.

Select the MVC Controller with views, using Entity Framework option and click Add.

On the next dialog use the drop downs to select a model class and a data context class. Then verify the controller name and click add.

When the process completes the following items will have been added to your project.

Controllers
 - ContactsController.cs
Views
 - Contacts
   - Create.cshtml
   - Delete.cshtml
   - Details.cshtml
   - Edit.cshtml
   - Index.cshtml
Add to nav bar

To add a link to the new section of the app to the nav bar open the _Layout.cshtml in the Views/Shared/ directory. The following is the section of the file that needs to be changed to add an item to the nav bar.

<ul class="nav navbar-nav">
    <li><a asp-area="" asp-controller="Home" asp-action="Index">Home</a></li>
    <li><a asp-area="" asp-controller="Contacts" asp-action="Index">Contacts</a></li>
    <li><a asp-area="" asp-controller="Home" asp-action="About">About</a></li>
    <li><a asp-area="" asp-controller="Home" asp-action="Contact">Contact</a></li>
</ul>

Specifically, the following line was added to provide access to the contact list page.

<li><a asp-area="" asp-controller="Contacts" asp-action="Index">Contacts</a></li>

Wrapping up

With the above, the application will be runnable. The code for this post can be found here. The next post in this series will cover the limitations of migrations when using SQLite with Entity Framework Core.

 

Entity Framework Core with SQLite Scaffolding Read More »

Entity Framework Core Errors Using Add-Migration

I started off trying to expand my sample from last week’s post and hit some issues when trying to add a migration for a new DbContext.

The Setup

I added the following DbContext that only has one DbSet and auto applies migrations in the constructor.

using EfSqlite.Models;
using Microsoft.EntityFrameworkCore;

public sealed class ContactsDbContext : DbContext
{
    private static bool _created;

    public DbSet<Contact> Contacts { get; set; }

    public ContactsDbContext(DbContextOptions<ContactsDbContext> options)
        : base(options)
    {
        if (_created) return;
        Database.Migrate();
        _created = true;
    }
}

The command

Using Visual Studio’s Package Manager Console I ran the following command.

Add-Migration AddContacts -Context ContactsDbContext
Error 1 – No parameterless constructor

The above command resulted in the following error.

No parameterless constructor was found on ‘ContactsDbContext’. Either add a parameterless constructor to ‘ContactsDbContext’ or add an implementation of ‘IDbContextFactory<ContactsDbContext>’ in the same assembly as ‘ContactsDbContext’.

I read the first sentence and added a parameterless constructor to ContactsDbContext. I did think it was strange that a parameterless constructor wasn’t required the other contexts I had written in the past, but the error said to add a parameterless constructor so that is what I did.

Error 2 – System.InvalidOperationException: No database provider has been configured for this DbContext

Now having a parameterless constructor I ran the Add-Migration command again and was greeted with the following error.

System.InvalidOperationException: No database provider has been configured for this DbContext. A provider can be configured by overriding the DbContext.OnConfiguring method or by using AddDbContext on the application service provider. If AddDbContext is used, then also ensure that your DbContext type accepts a DbContextOptions<TContext> object in its constructor and passes it to the base constructor for DbContext.

The second error forced me to step back and think more about what the problem was as it didn’t have an action I could take as the first sentence, which is, of course, my fault for not fully digesting what the error was saying.

The fix

The bit I was missing was the fact that I hadn’t added the following to the ConfigureServices function of the project’s Startup class.

services.AddDbContext<ContactsDbContext>(options =>
    options.UseSqlite(Configuration.GetConnectionString("Sqlite")));

With the above added I removed the parameterless constructor from ContactsDbContext and was able to successfully run the add migration command again.

Wrapping up

The moral of the story is to actually read the full error message before running off and trying to fix the problem. The second error message saying “using AddDbContext on the application service provider” is what triggered me to head in the right direction.

This was also a good reminder that tools like the ones used by Add-Migration can/do compile the project they are being used on in order to have enough context to perform their tasks.

Entity Framework Core Errors Using Add-Migration Read More »

Entity Framework Core with SQLite

All the applications used as examples for ASP.NET Core and Entity Framework Core from this site so far used database running SQL Server/SQL Express. In addition to the Microsoft-based SQL databases, Entity Framework Core has support for a number of other database providers. This post is going to look at using SQLite. A full list of the support database providers can be found here.

Starting point

Using Visual Studio 2017 I started with a new ASP.NET Core project using Individual User Accounts which ensured all the Entity Framework Core bits were present. The template in RC 4 used packages based on the Core 1.0.3 which I upgraded to 1.1.0. The project at this point can be found here.

Just a side note this project was created when Visual Studio 2017 was at the RC 4 stage. This code associated with this post will be updated when Visual Studio 2017 is released.

Naming warning

As you will be able to see with the structure of the solution I started this work using the project name SQLite. With this project name, it was impossible to get the SQLite package to install. If you see something like the following renaming your project should get you running.

Cycle detected: 
   Sqlite (>= 1.0.0) -> Microsoft.EntityFrameworkCore.Sqlite (>= 1.1.0) -> Microsoft.Data.Sqlite (>= 1.1.0) -> SQLite (>= 3.13.0)

This issue is where I found out what the problem was.

Add SQLite Packages

Right-click on the project file and click Manage NuGet Packages.

Select Browse and in the search box enter “Microsoft.EntityFramework.Sqlite” and install the two packages that are found.

Remove SqlServer Packages

While still in the Manage NuGet Packages screen click on the Installed tab. Select and uninstall the following packages.

Microsoft.EntityFrameworkCore.SqlServer
Microsoft.EntityFrameworkCore.SqlServer.Design

Configuration changes

Open appsettings.json and in the ConnectionStrings delete the line for DefaultConnection. Next, in the same section add a line for a SQLite connection string. The following is the result.

"ConnectionStrings": {
  "Sqlite": "Data Source=Database.db"
}

The above will expect the database file to be in the same location as the application is running. For a debug build the database file can be found in the \bin\Debug\netcoreapp1.0\ directory of the project.

Startup changes

The final location to change is in the ConfigureServices function of the Startup class. The following shows the addition of the application DB context before and after the changes.

Before:
services
  .AddDbContext<ApplicationDbContext>(options =>  options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection")));
 
After:
services
  .AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlite(Configuration.GetConnectionString("Sqlite")));

Wrapping up

The application is now runnable using SQLite as its backing data store. At this point, the only thing using data access is related to identity. The first time an attempt is made to access the database you may be prompted to apply migrations.

I have been using SQLite Studio to view the data in my database if you have that need outside of the application it does a good job.

The code in its final state can be found here.

Entity Framework Core with SQLite Read More »

Creating and Connecting to a Virtual Machine in Azure

To get ready for the release of Visual Studio 2017 and the finalization of the ASP.NET Core’s move from project.json back to csproj I have been using a virtual machine (VM) in Azure instead of a local install. I did this just to avoid any issues as I wanted to keep my projects on project.json until the tooling was ready to go.

Since I create VMs infrequently it takes me a couple of minutes to remind myself of the steps I need to use. As stated above this post is going to be using Microsoft Azure and if you don’t have an account you can sign up for a free one here.

Creating a virtual machine

Starting from the portal click the plus button in the top left.

Next either browse the list for the type of VM you are looking to create or use the search. For this example use the search box and type “Visual Studio 2017” and press enter.

This will bring up a list of VMs that come with Visual Studio 2017 already installed. For this example, I will be using the VM with Visual Studio Enterprise 2017 RC on Windows 10 Enterprise N (x64).

When you select a VM on the screen above it will add a new blade with information about the VM. If the VM is what you are looking for then you can click the Create button.

Next, there will be a series of steps to collect information and settings needed to create the VM. The first is the basic setting where the VM will be named, given a username, password, resource group, and data center location. After finishing the settings click OK.

Next, is VM size selection. I am just using one of the recommend, but if you need more options click View All. Click the size you want to use and then click the Select button.

Next, is more settings. I just took the defaults and clicked OK.

The final screen is a validation and summary. If all looks good on this screen click the OK button to kick off the creation of the VM.

You will now be back on your dashboard page which will contain a new tile showing that your new VM is being deployed.

Connected to your VM

When deployment is finished the portal will automatically redirect you to the new VM’s page.

Click the Connect button in the top center of the page which will trigger the download of a VS2017Blog.rdp. Open the file that was downloaded which a Remote Desktop Connection. With the following warning. Click Connect.

Next is the credentials screen. If you are logged on to a domain you will need to click More choices and then Use a different account which will leave you on a screen like the following. The first box is domain\username which in this case is VS2017Blog\vs2017blog. Sorry for the confusion I should have used different values during the creation of the VM. The second box is the password. Click OK.

Finally, you will see a certificate warning. I believe this happens because it is created as part of the VM creation process and not from an authority. Click Yes and you will be logged into your VM.

Wrapping up

Now that you have a VM up and running you can do anything you need to. For me, it is running the RC version of Visual Studio 2017 without risking having any compatibility issues. By the time this post is out Visual Studio 2017 will have been released.

The VM create above has been deleted which is why I didn’t attempt to hide the IP address.

Creating and Connecting to a Virtual Machine in Azure Read More »

Visual Studio 2017 error encountered while cloning the remote Git repository

Going through the process of getting Visual Studio 2017 installed on all my machines has been pretty smooth. The new installer works great and makes it much clearer what needs to be installed.

The issue

The one issue I have had, which was only an issue on one install, is an error when trying to clone a repo from GitHub. I say GitHub but really it would be a problem with any Git repo. The following is the error I was getting.

Error encountered while cloning the remote repository: Git failed with a fatal error.
CloneCommand.ExecuteClone

The solution

After searching I found a lot of things to try. I uninstalled and reinstalled Git for Windows multiple times using both the Visual Studio Installer and the stand alone installer. I finally stumbled onto this forum thread which had a solution that worked for me. The following is a quote of the reason for the issue and a fix posted by J Wyman who is a software engineer for Microsoft’s Developer Division.

After intensive investigation we were able to determine that Git was accidentally loading an incorrect version of libeay32.dll and ssleay32.dll due to library loading search order and precedence rules. Multiple users have confirmed that copying the binaries from the “<VS_INSTALL>\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer\Git\mingw32\bin\” folder to the “<VS_INSTALL>\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer\Git\mingw32\libexec\git-core\” folder completely resolved the issue for him.

Any other users seeing similar problem should attempt the same solution.

I hope this gets other people up and running as it did me. My only worry about this fix is what happens with Git gets updated.

Visual Studio 2017 error encountered while cloning the remote Git repository Read More »

ASP.NET Core Conversion to csproj with Visual Studio 2017 and update to 1.1.1

On March 7th Visual Studio 2017 was released bring the ASP.NET Core tools preview. ASP.NET Core 1.1.1 was also released. This post is going to cover converting the project from my MailGun post from being project.json based to csproj as well as migrating from the project from ASP.NET Core 1.0.2 to 1.1.1. Here is the project as it stood before I made any changes.

Visual Studio 2017

The first step is to get a version of Visual Studio 2017 (VS 2017) installed. The download page can be found here. Make sure to grab the community edition if you are looking for a free fully-featured IDE option. Check out this blog post from Microsoft on the many new features Visual Studio 2017 brings.

The installer for VS 2017 has changed a lot from previous versions. The way it works now is you select the workload you use and it only installs the bit it has to to keep the size of install down. The following is a screen shot from my install. I have a lot more workloads checked that is needed for just an ASP.NET Core application. At a minimum make sure the “ASP.NET and web development” workload gets installed. If you are interested in cross-platform development scroll to the bottom and also check “.NET Core cross-platform development”.

Project conversion

When you open the solution in VS 2017 it will prompt you to do a one-way upgrade.

After the conversion is complete a migration report will open. Below is mine. I had no issues, but if there were any this report should give you some idea of how they should be addressed.

As part of the conversion process, the following file changes happened.

Deleted:
global.json
ASP.NET-Core-Email.xproj
project.json

Added:
ASP.NET-Core-Email.csproj

That is all there is to the conversion. The tooling takes care of it all and your project should keep work just as before. The sample project post conversion can be found here.

Migration from 1.0.x to 1.1.1

The migration is almost as simple as the project conversion. In the solution explorer right click on the project to be migrated and select Properties.

Find the Target framework selection and select .NETCoreApp 1.1. Then save your solution.

Next, open the NuGet Package Manager. It can be by right click on the project and selecting Manage NuGet Packages or from the Tools > NuGet Package Manager > Manage NuGet Packages for Solution.

Select the Updates tab and update all the related packages to 1.1.1 and click the Update button.

If you want a specific list of all the package changes check out the associated commit.

The only other change needed is in the constructor of the Startup class.

Before:
builder.AddUserSecrets();

After:
builder.AddUserSecrets<Startup>();

Wrapping up

After all the changes above your solution will be on the latest released bits. Having been following releases since beta 4 I can tell you this is one of the easiest migration so far. I may be partial, but .NET and Microsoft seem to be getting better and better over the last couple of years.

I am going to leave you with a few related links.

ASP.NET Core 1.1.1 Release Notes
Announcing New ASP.NET Core and Web Dev Feature in VS 2017
Project File Tools – Extension for IntelliSense in csproj
Razor Language Services – Extension for tag helper Intellisense

ASP.NET Core Conversion to csproj with Visual Studio 2017 and update to 1.1.1 Read More »

Two Years of Blogging

Wednesday will mark two years for this blog. This post is going to be a bit of a retrospective on my blogging journey so far.

Positives

A number of positive things have come out of maintaining this blog some of them I expected other were surprises.

  • Driver for learning new things
  • Opportunity to use new/different technology outside of my normal work
  • Helping people learn a new concept or get past a sticking point
  • New connections with people outside of my normal circles
  • Improvements to my written communication skills
  • Improvements in picking up new technology

Challenges

In addition to the above, the following is a list of things that challenged me. Some of them have helped me grow and others continue to give me problems.

  • Learning new things on a deadline
  • Self-applied pressure to meet my goal of a post a week
  • Not focusing on stats, shares, comment, etc.
  • Picking the right things to learn
  • Losing focus

Hosting challenges

I hit quite a few issues with my host this year. The biggest being multiple multi-day downtimes, high site load times, and difficult to implement SSL support.

In an attempt to address the second two problems I started using Cloudflare which worked well, but the lack of SSL support from my host cause some quirky issues.

All this led to me switching to NodeHost last month. That is a referral link that will give you a $5 credit when you sign up. NodeHost does all their hosting on SSDs and has built-in support for Let’s Encrypt which took care of my SSL problems. Getting a site setup initially was harder that on my old host, but it was totally worth the switch and ended up costing less.

Even with the hosting change I still recommend using Cloudflare. By using Cloudflare my host is having to handle around 50% few requests thanks to Cloudflare’s caching.

Top post of the year

It is interesting to look back and see which post have done the best over time. It is surprising to me that my first post is still in the top 5 post for the last year. One of my top posts was featured on ASP.NET community spotlight which always pushes way more traffic.

The next year

From where it stands now I see more than enough things in the ASP.NET Core/Aurelia/Angular areas to learn and they will continue to be my focus over the next year. I am especially excited that ASP.NET Core tooling to be complete and for Visual Studio 2017 to be released this year. I am also really looking forward to the .NET Standard 2 support to be released. With those two items taken care of, I will feel completely comfortable recommending ASP.NET Core it a much wider range of scenarios.

As always I am open to topic suggestions for topics and happy to answer questions. Thank you for the support of the last couple of years.

Two Years of Blogging Read More »

Log Requests and Responses in ASP.NET Core

Note: An updated version of this post for ASP.NET Core 3 and above is available.

As part of trying to do some debugging, I needed a way to log the requests and responses. Writing a piece of middleware seemed to be a good way to handle this problem. It also turned out to be more complicated that I had expected to deal with the request and response bodies.

Middleware

In ASP.NET Core middleware are the components that make up the pipeline that handles request and responses for the application. Each piece of middleware called has the option to do some processing on the request before calling next piece of middleware in line. After execution returns from the call to the next middleware, there is an opportunity to do processing on the response.

The pipeline for an application is set in the Configure function of the Startup class. Run, Map and Use are the three types of middleware. Run should only be used to terminate the pipeline. Map is used for pipeline branching. Use seems to be the most common type of middleware that does some processing and call the next middleware in line. For more detail see the official docs.

Creating Middleware

Middleware can be implemented as a lambda directly in the Configure function, but more typically it is implemented as a class that is added to the pipeline using an extension method on IApplicationBuilder. This example will be going the class route.

This example is a piece of middleware that using ASP.NET Cores built-in logging to log requests and responses. Create a class called RequestResponseLoggingMiddleware.

The class will need a constructor that takes to arguments both will be provided by ASP.NET Core’s dependency injection system. The first is a RequestDelegate for the next piece of middleware to be called. The second is an ILoggerFactory which will be used to create a logger. The RequestDelegate is stored to the class level _next variable and the loggerFactory is used to create a logger which is stored to the class level _logger variable.

public class RequestResponseLoggingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger _logger;

    public RequestResponseLoggingMiddleware(RequestDelegate next,
                                            ILoggerFactory loggerFactory)
    {
        _next = next;
        _logger = loggerFactory
                  .CreateLogger<RequestResponseLoggingMiddleware>();
    }
}

Add an Invoke function which is the function that will be called when your middleware is run by the pipeline. The following is the function that does nothing other than call the next middleware in the pipeline.

public async Task Invoke(HttpContext context)
{
     //code dealing with request

     await _next(context);

     //code dealing with response
}

Next, add a static class to simplify adding the middleware to the application’s pipeline. This is the same pattern the built-in middleware uses.

public static class RequestResponseLoggingMiddlewareExtensions
{
    public static IApplicationBuilder UseRequestResponseLogging(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<RequestResponseLoggingMiddleware>();
    }
}

Adding to the pipeline

To add the new middleware to the pipeline open the Startup to the Configure function and add the following line.

app.UseRequestResponseLogging();

Keep in mind that the order in which middleware is added can make a difference in how the application behaves. Since the middleware this post is dealing with is logging I have placed it near the begging of the pipeline just before app.UseStaticFiles().

Logging requests and responses

Now that the setup work for our new middleware is done we will come back to its Invoke function. As I stated above this ended up being more complicated that I expected, but thankfully I found this by Sul Aga which really helped me work through the issues I was having.

I created a couple of helper functions that we will look at first. The following is the function call to create the string that will be logged for a request.

private async Task<string> FormatRequest(HttpRequest request)
{
    var body = request.Body;
    request.EnableRewind();

    var buffer = new byte[Convert.ToInt32(request.ContentLength)];
    await request.Body.ReadAsync(buffer, 0, buffer.Length);
    var bodyAsText = Encoding.UTF8.GetString(buffer);
    request.Body = body;

    return $"{request.Scheme} {request.Host}{request.Path} {request.QueryString} {bodyAsText}";
}

The key to getting this function to work and allow reading of the request body was request.EnableRewind() which allows us to read from the beginning of the stream. The rest of the function is pretty straight forward.

The next function is used to get the string to that will be used to log the response body. This function looks simpler than it is and only works because of how it is called from the Invoke function.

private async Task<string> FormatResponse(HttpResponse response)
{
    response.Body.Seek(0, SeekOrigin.Begin);
    var text = await new StreamReader(response.Body).ReadToEndAsync(); 
    response.Body.Seek(0, SeekOrigin.Begin);

    return $"Response {text}";
}

Finally, the Invoke which does the logging and jumps through some hoops to allow the response body to be read.

public async Task Invoke(HttpContext context)
{
    _logger.LogInformation(await FormatRequest(context.Request));

    var originalBodyStream = context.Response.Body;

    using (var responseBody = new MemoryStream())
    {
        context.Response.Body = responseBody;

        await _next(context);

        _logger.LogInformation(await FormatResponse(context.Response));
        await responseBody.CopyToAsync(originalBodyStream);
    }
}

As you can see the trick to reading the response body is replacing the stream being used with a new MemoryStream and then copying the data back to the original body steam. This works and is a concept I found in Sul’s blog post. I don’t know how much the affect performance and would make sure to study how it scales or just avoid using it in production as much as possible.

Wrapping up

This entry didn’t turn out anything like I expected. I came into this looking to do a very simple post due to some time restrictions and it turned into something larger. I hope you find it helpful. I didn’t do a full repo for this week’s post, but you can a gist with the middleware classes here.

Log Requests and Responses in ASP.NET Core Read More »

Email with ASP.NET Core Using Mailgun

Sending emails from ASP.NET Core using Mailgun is a topic I covered in this post almost a year ago. The previous post was before ASP.NET Core hit 1.0 and I didn’t save or upload the code to GitHub. Based on the comments on the post I decided to redo the post using the current version of ASP.NET.

Starting point

For this post, I created a new ASP.NET Core Web Application targeting .NET Core using Individual User Accounts for authentication. The project before any changes for email can be found here.

UI and Controller change to support email

The project template comes with all the UI and controller functions need to support email, but they are commented out. The following is going to walk through uncommenting the proper code.

Account controller

Starting with the Register function the code to send a confirmation email needs to be uncommented and the existing call to _signInManager.SignInAsync should be commented out to keep a user from being signed in before their email address has been confirmed. The following is after the changes.

var code = await _userManager.GenerateEmailConfirmationTokenAsync(user);
var callbackUrl = Url.Action("ConfirmEmail", "Account", 
                             new { userId = user.Id, code = code }, 
                             protocol: HttpContext.Request.Scheme);
await _emailSender.SendEmailAsync(model.Email, "Confirm your account",
   $"Please confirm your account by clicking this link: 
     <a href='{callbackUrl}'>link</a>");
//await _signInManager.SignInAsync(user, isPersistent: false);

Next, in the Login function add a check to verify a user’s account has been confirmed before allowing them to sign in. The new code starts with var user = await _userManager.FindByNameAsync(model.Email); the code above it is just to provide context.

 public async Task<IActionResult> Login(LoginViewModel model, 
                                        string returnUrl = null)
 {
      ViewData["ReturnUrl"] = returnUrl;
      if (ModelState.IsValid)
      {
          var user = await _userManager.FindByNameAsync(model.Email);
          if (user != null)
          {
              if (!await _userManager.IsEmailConfirmedAsync(user))
              {
                  ModelState.AddModelError(string.Empty, 
                              "You must have a confirmed email to log in.");
                  return View(model);
              }
          }

Finally, in the ForgotPassword function uncomment the following to enable sending the user a password reset link.

var code = await _userManager.GeneratePasswordResetTokenAsync(user);
var callbackUrl = Url.Action("ResetPassword", "Account", 
                             new { userId = user.Id, code = code }, 
                             protocol: HttpContext.Request.Scheme);
await _emailSender.SendEmailAsync(model.Email, "Reset Password",
   $"Please reset your password by clicking here: <a          
     href='{callbackUrl}'>link</a>");
return View("ForgotPasswordConfirmation");
Forgot password view

To enabled the UI related to sending an email for a forgotten password open ForgotPassword.cshtml found in the Views/Account/ directory and uncomment the following.

<form asp-controller="Account" asp-action="ForgotPassword" 
      method="post" class="form-horizontal">
    <h4>Enter your email.</h4>
    <hr />
    <div asp-validation-summary="All" class="text-danger"></div>
    <div class="form-group">
        <label asp-for="Email" class="col-md-2 control-label"></label>
        <div class="col-md-10">
            <input asp-for="Email" class="form-control" />
            <span asp-validation-for="Email" class="text-danger"></span>
        </div>
    </div>
    <div class="form-group">
        <div class="col-md-offset-2 col-md-10">
             <button type="submit" 
                     class="btn btn-default">Submit</button>
        </div>
    </div>
</form>

Warning for sites with existing users

The changes above will cause issues for any existing users since they will not have completed the email confirmation step keeping them from being able to log in or reset passwords. Manually marking existing users as confirmed can be done by updating the EmailConfirmed bit field to true in the AspNetUsers table.

Mailgun

Mailgun is an email service run by Rackspace that provides a simple API for sending emails. The free level of the service allows up to 10k emails to be sent a month.

You can sign up for an account here. Once logged in go to the Domains section.

Next, select your domain should only be one if you are on a new account. This will take you to a screen that looks like the following some of which will be needed to connect with the Mailgun API. I took the time to replace my information with a fake version so this screen shot could be referenced using the values from the screenshot for the rest of the post.

Configuration

Settings class

In order to hold and load Mailgun email related settings add a new EmailSettings class. In the sample project, this class can be found in the Configuration directory. The following is the full contents of the file.

public class EmailSettings
{
    public string ApiKey { get; set; }
    public string ApiBaseUri { get; set; }
    public string RequestUri { get; set; }
    public string From { get; set; }
}
User secrets introduction

User secrets is a concept in ASP.NET Core used to set configuration items and have them stored outside of the project so they will be excluded from version control. They are a great way to store private API key and related items which is why they will be used to store our Mailgun configuration items. I will be coving the basics here, but for a more detail explanation check out the official docs on the topic of app secrets.

Setting user secrets

In the Solution Explorer right-click on the project and select Manage User Secrets.

This will open the secrets.json file which will be used to store secrets related to the select project. Keep in mind this file is stored in your user directory in an unencrypted way so don’t view it as a secured store.

Based on the screenshot above from Mailgun’s domain detail page the json file would look like the following. The RequestUri is the only setting not pulled from the domain settings above and would just need fakesandbox replaced with the sandbox ID for your domain.

{
  "EmailSettings": {
    "ApiKey": "api:key-fakeapikey",
    "ApiBaseUri": "https://api.mailgun.net/v3/",
    "RequestUri": "fakesandbox.mailgun.org/messages",
    "From": "[email protected]"
  }
}
Loading user secrets in Startup

In the ConfigureServices function of the Startup class the EmailSettings section of our user secrets can be loaded and made available via the dependency injection system using the following line of code.

services.Configure<EmailSettings>(Configuration.GetSection("EmailSettings"));

Not that user secrets are only meant to be used for development and for a production build of the applications the settings would need to be moved to a different location such as environment variables or Azure Key Vault.

Using Mailgun to send email

Not that the application has the email sending portion of the UI enabled the SendEmailAsync function of the AuthMessageSender class needs to be implemented. The class can be found in the MessageServices.cs file of the Services directory.

Injection of email settings

The first change needed is to add a class level variable to store email settings and to add a constructor that will allow the email setting to be injected.

private readonly EmailSettings _emailSettings;

public AuthMessageSender(IOptions<EmailSettings> emailOptions)
{
    _emailSettings = emailOptions.Value;
}
Sending an email

The body of the SendEmailAsync function is where the call to Mailgun’s API will be made using the email setting injected via the class’s constructor. The following is the full body of the function.

using (var client = new HttpClient { BaseAddress = 
                                     new Uri(_emailSettings.ApiBaseUri) })
{
    client.DefaultRequestHeaders.Authorization = 
      new AuthenticationHeaderValue("Basic",
    Convert.ToBase64String(Encoding.ASCII.GetBytes(_emailSettings.ApiKey)));

    var content = new FormUrlEncodedContent(new[]
    {
        new KeyValuePair<string, string>("from", _emailSettings.From),
        new KeyValuePair<string, string>("to", email),
        new KeyValuePair<string, string>("subject", subject),
        new KeyValuePair<string, string>("html", message)
    });

    await client.PostAsync(_emailSettings.RequestUri, 
                           content).ConfigureAwait(false);
}

The Mailgun API key is sent as an authentication header value with the rest of the parameters being sent via form URL encoded content. Finally, the Request URI from the email settings is used to send the post request to Mailgun.

If you just need to send a plain text instead of HTML the “html” key can be replaced with “text”.

Authorized recipients required for test domain

When using Mailgun with the default test domain note that only emails addressed to Authorized Recipients will be delivered. To add a recipient click Authorized Recipients button on Mailgun’s Domains page.

This will take you to the Authorized Recipients page where you can use the Invite New Recipient button to add a recipient.

Enter the email address you want to add and click the Send Invite button. After the email address is confirmed mail to that address will be delivered. Keep in mind this is only for test accounts and doesn’t have to be done when being used with a real domain.

Using logs to verify state of emails

During my testing, I wasn’t seeing emails come through and I thought something was wrong with my code, but it turned out that Mailgun was getting the request to send the mail they just hadn’t be processed yet. The Logs section of your Mailgun account is helpful for determining if they are getting your request to send an email or not.

As you can see in the screenshot the email request was accepted, but not yet delivered. It was 10 minutes later before the email was actually delivered. I am not sure if this delay is just for test domains or if would apply to live ones as well.

Other Email Options

Mailgun is obviously not the only option for sending emails. This post from Mashape lists 12 API providers. In addition SMTP is also an option which this post by Steve Gordon covers.

Wrapping up

The code finished code that goes with this post can be found here. Thank you to all the commenters on the original post for stepping in when I didn’t have all the answer or code available.

Email with ASP.NET Core Using Mailgun Read More »