Intro

I'm used to be using async/await keywords in .NET. It increases the throughput of the services I'm working on dramatically.  At some point, using async gained me 10x performance improvements in terms of users throughput, by simply applying asynchronous overload to the critical part of a software. But if it's so simple to use and at the same time so powerful - how does it work behind the scenes? Well, not that simple. The keywords I love so much for their simplicity are just syntactic sugar and translated on a compiler level to something called Async State Machine.

Example

Let's dive a bit deeper and rely on an example. I will use simple program that tries to get the data from
Dictionary if the key is found, if not I await with
Task.Delay and after that initializing it for later use.

That's how I see the code in IDE:

private readonly Dictionary<string, decimal> _dictionaryCache = new Dictionary<string, decimal>();

public Task<decimal> GetValueFromCacheAsync(string key)
{
    if(_dictionaryCache.ContainsKey(key))
    {
        return Task.FromResult(_dictionaryCache[key]);
    }

    return InitCacheLazyAsync(key);
}

private async Task<decimal> InitCacheLazyAsync(string key)
{
    await Task.Delay(1000);
    var value = 11.11m;

    _dictionaryCache.Add(key, value);

    return value;
}

And that's what the code looks like on a compiler level:

using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Threading.Tasks;

class StateMachineCompiler
{
    public Task<decimal> GetValueFromCacheAsync(string key)
    {
        return _dictionaryCache.ContainsKey(key) ? Task.FromResult(_dictionaryCache[key]) : InitCacheLazyAsync(key);
    }

    private Task<decimal> InitCacheLazyAsync(string key)
    {
        var stateMachine = new InitCacheLazyStateMachine();
        stateMachine.RelatedTo = this;
        stateMachine.Builder = AsyncTaskMethodBuilder<decimal>.Create();
        stateMachine.State = State.Created;
        stateMachine.Key = key;
        stateMachine.Builder.Start(ref stateMachine);

        return stateMachine.Builder.Task;
    }

    private struct InitCacheLazyStateMachine : IAsyncStateMachine
    {
        public State State;
        public AsyncTaskMethodBuilder<decimal> Builder;
        public string Key;
        public StateMachineCompiler RelatedTo;
        private TaskAwaiter _taskAwaiter;

        // Task state management.
        void IAsyncStateMachine.MoveNext()
        {
            // ...
        }
    }

    enum State 
    {
        //...
    }
}

Wow! No more async await keywords in our code, but does it mean we're running synchronous code? As always in software engineering it depends on a context.

By default .NET runtime will try to execute the Task straight away if it could - synchronously. The reason for that is to not overhead with queuing and scheduling management if the code is actually can perform synchronously. But even on synchronous method marked with async (without await inside) state machine will be generated anyway and the code which could potentially be inlined and executed much faster will be generating additional complexity for state machine management. That's why it's so important to mark methods as async if and only if there is await inside.

So let's take a closer look at State Machine compiler generated contract:

public interface IAsyncStateMachine
{
    //
    // Summary:
    //     Moves the state machine to its next state.
    void MoveNext();
    //
    // Summary:
    //     Configures the state machine with a heap-allocated replica.
    //
    // Parameters:
    //   stateMachine:
    //     The heap-allocated replica.
    void SetStateMachine(IAsyncStateMachine stateMachine);
}

The main method state machine operates on is
void MoveNext(); An interesting part that it returns nothing because mutates the state by reference. By design it's relying on awaiters completion and what it actually does is well described in docs:
Moves the state machine to its next state.

StateMachine itself may have three different states to track Task status:

enum State
{
    Completed = -2,
    Created = -1,
    Awaiting = 0,
}

Once AsyncTaskMethodBuilder hits Start command state machine's
MoveNext method is being executed.

What MoveNext does is state management, it checks if awaiter is completed or not and changes the state of state machine and/or Task. Once state machine is finally able to mark the task as completed it calls
Builder.SetResult(); and it's done.

Here is a basic flow for this example:

GetValueFromCacheAsync is now initializing state machine (2) and starts Task processing/scheduling (3) for the first time.

Once task processing's being started the runtime checks if task can be executed synchronously and if so Task.ExecuteEntry is called, Task is marked as completed, and we got the task fulfilled with result straight away.
If the task can't run synchronously - that's where by default ThreadPool queuing comes in play.

ThreadPool queuing operates on events and callbacks as well as asynchronous operations. There are two types of processing - CPU bound and I/O bound. On truly asynchronous operations such as writing to Disk or making Network requests .NET waits to be notified that native process is completed and assigns a thread to continue processing the result of a native service. That's what called I/O bound operations. For CPU bound there could be a thread to process the result in a background. For a better understanding of how this is possible I recommend to read article by Stephen Cleary - there is no thread.

So once ThreadPool is notified about completion of asynchronous operation it notifies Async State Machine again and MoveNext() is executed second time.

MoveNext checks if awaiter is completed and since it is - sets the result of the task and it's done. The state machine exits and .NET continue processing.

private struct InitCacheLazyStateMachine : IAsyncStateMachine
{
    public State State;

    public AsyncTaskMethodBuilder<decimal> Builder;

    public string Key;

    public StateMachineCompiler RelatedTo;

    private TaskAwaiter _taskAwaiter;

    public void SetStateMachine(IAsyncStateMachine stateMachine)
    {
        // Obsolete contract.
    }

    void IAsyncStateMachine.MoveNext()
    {
        State currentState = State;
        decimal result;
        try
        {
            TaskAwaiter awaiter;
            if (currentState != State.Awaiting)
            {
                awaiter = Task.Delay(1000).GetAwaiter();
                if (!awaiter.IsCompleted)
                {
                    State = State.Awaiting;
                    _taskAwaiter = awaiter;
                    var stateMachine = this;

                    // Schedules the state machine to proceed to the next action when the specified awaiter completes.
                    Builder.AwaitUnsafeOnCompleted(ref awaiter, ref stateMachine);

                    return;
                }
            }
            else
            {
                awaiter = _taskAwaiter;
                _taskAwaiter = new TaskAwaiter();
                State = State.Created;
            }

            awaiter.GetResult();
            result = new decimal(1111, 0, 0, false, 2);
            RelatedTo._dictionaryCache.Add(Key, result);
        }
        catch (Exception ex)
        {
            State = State.Completed;
            Builder.SetException(ex);

            return;
        }

        State = State.Completed;
        Builder.SetResult(result);
    }
}

Conclusion

• Do not mark methods async if there is no await inside.
• The code we write so tidy and clean could be a mess behind the scenes - it's always good to strive for knowledge and understand the underlying complexity of a software.

I haven't touched in this article error handling parts as well as I didn't want to dive too deeply inside. I may touch it in my future posts, though.
Please feel free to join the discussion or give me a heads up on any part you're interested in.

References:

1. TaskScheduler QueueTask GitHub link
2. Task ExecuteEntry GitHub link
3. There is no thread