Styles of Asynchronous API

Showcases different variations of asynchronous APIs with examples of using libcurl, specifically, doing 2 GET requests - both sequentially and concurrently.

NO threads and/or multithreading involved to disconnect any associations of coroutines or fibers with threads. Something is intentionally simpler, while still having as much details as possible.

1 # introduction

Lets begin with simple C-style API on top of libcurl C API and build a program that may look like this:

// our CURL API
std::string CURL_get(const std::string& url);

int main()
{
    const std::string r1 = CURL_get("localhost:5001/file1.txt");
    const std::string r2 = CURL_get("localhost:5001/file2.txt");
    return int(r1.size() + r2.size()); // handle results
}

Next, lets have idiomatic C-style callbacks API, intentionally, not C++ one, see the note, to run requests concurrently:

// libcurl bookkeeping
using CURL_Async = void*;
CURL_Async CURL_async_create();
void CURL_async_destroy(CURL_Async curl_async);
void CURL_async_tick(CURL_Async curl_async);

// main async callback API
void CURL_async_get(CURL_Async curl_async
    , const std::string& url
    , void* user_data
    , void (*callback)(void* user_data, std::string response));

int main()
{
    struct State
    {
        int count = 0;
        std::string r1;
        std::string r2;
    };

    CURL_Async curl_async = CURL_async_create();
    State state;
    CURL_async_get(curl_async, "localhost:5001/file1.txt", &state
        , [](void* user_data, std::string response)
    {
        State& state = *static_cast<State*>(user_data);
        state.count += 1;
        state.r1 = std::move(response);
    });
    CURL_async_get(curl_async, "localhost:5001/file2.txt", &state
        , [](void* user_data, std::string response)
    {
        State& state = *static_cast<State*>(user_data);
        state.count += 1;
        state.r2 = std::move(response);
    });
    while (state.count != 2) // wait for 2 requests to finish
    {
        CURL_async_tick(curl_async);
    }
    CURL_async_destroy(curl_async);
    return int(state.r1.size() + state.r2.size());
}

There is a need to have a State for bookkeeping, pass it as void* user data to access later and, finally, run an event loop to give libcurl a chance to process requests. Note, however, requests execute concurrently now, as in 2 requests are active at the same time.

2 # setup with cmake + libcurl

git clone https://github.com/microsoft/vcpkg
cd vcpkg
bootstrap-vcpkg.bat
:: for a later use, assume we are in `K:\vcpkg`
set VCPKG_ROOT=K:\vcpkg
:: to make `vcpkg` available
set path=%VCPKG_ROOT%;%PATH%

For the project (which is going to have async_api_styles name), vcpkg manifest mode is used. Together with curl setup, all required steps are:

cd async_api_styles
vcpkg new --application
vcpkg add port curl

Note: to find exact curl package name, vcpkg search curl was used which prints:

cmake_minimum_required(VERSION 3.24 FATAL_ERROR)
project(async_api_styles LANGUAGES CXX)

add_executable(00_cmake_libcurl main.cc)
find_package(CURL REQUIRED)
target_link_libraries(00_cmake_libcurl PRIVATE CURL::libcurl)

find_package(CURL REQUIRED) syntax together with CURL::libcurl target name is found from the output log of vcpkg install curl (or during CMake configuration run) which prints:

curl is compatible with built-in CMake targets:

    find_package(CURL REQUIRED)
    target_link_libraries(main PRIVATE CURL::libcurl)

#include <curl/curl.h>

int main()
{
    CURL* curl = curl_easy_init();
    assert(curl);
    curl_easy_cleanup(curl);
}

cd async_api_styles
cmake -S . -B build ^
  -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%\scripts\buildsystems\vcpkg.cmake
cmake --build build --config Debug
:: run test
./build/Debug/00_cmake_libcurl.exe

3 # building blocking API

Our blocking, synchronous API for GET request is straightforward, lets go with function that looks like this:

std::string CURL_get(const std::string& url);

libcurl comes with two different APIs, “easy” and “multi”. Lets use easy interface; libcurl examples available online, including official simple.c example for a start.

Everything together leads to the implementation below, where curl_easy_perform() call is the main one that blocks the execution until request complete; once complete, we can return results:

#include <string>

#include <curl/curl.h>

#if defined(NDEBUG)
#  undef NDEBUG
#endif
#include <cassert>

static size_t CURL_OnWriteCallback(void* ptr, size_t size, size_t nmemb, void* data)
{
    std::string& response = *static_cast<std::string*>(data);
    response.append(static_cast<const char*>(ptr), size * nmemb);
    return (size * nmemb);
}

std::string CURL_get(const std::string& url)
{
    CURL* curl = curl_easy_init();
    assert(curl);

    CURLcode status = curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
    assert(status == CURLE_OK);
    status = curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
    assert(status == CURLE_OK);
    
    std::string response;
    status = curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CURL_OnWriteCallback);
    assert(status == CURLE_OK);
    status = curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
    assert(status == CURLE_OK);

    status = curl_easy_perform(curl);
    assert(status == CURLE_OK);

    long response_code = -1;
    status = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);
    assert(status == CURLE_OK);
    assert(response_code == 200L);

    curl_easy_cleanup(curl);
    return response;
}

Note on error handling: for now, we crash on any unexpected error - as in “crash the whole application”. assert() is enabled always intentionally to simplify both, the sample code and debugging:

// after all includes, main.cc
#if defined(NDEBUG)
#  undef NDEBUG
#endif
#include <cassert>

This is “bad” for generic, low-level library API/code, but could be fine sometimes. We’ll discuss error handling later.

#include <print>

int main()
{
    const std::string r = CURL_get("localhost:5001/file1.txt");
    std::println("CURL_get(file1.txt): '{}'", r);
}

that.. should crash since we don’t have local HTTP server running to serve localhost:5001/file1.txt. See the next section on how to make it happen.

3.1 # run simple http server for tests

To run sample code, lets use Python to have simple HTTP server that hosts files in the current directory, see serve.cmd:

python -m http.server 5001

Given the directory that has file1.txt and file2.txt, CURL_get("localhost:5001/file1.txt") and CURL_get("localhost:5001/file2.txt") should work and return the content of the files, see blocking libcurl section.

4 # building classic C-style callbacks API

4.1 # thoughts on the design

Now, lets imagine simplest possible asynchronous API. The difference to blocking API is that we ask the system to start a GET request and the response should arrive some time later. The system invokes a user-provided callback to notify us once everything is done:

void CURL_async_get(const std::string& url
    , void (*callback)(std::string));

// start a request:
CURL_async_get("localhost:5001/file2.txt"
    , [](std::string response)
{
    // probably, some time later:
    std::println("got response: {}", response);
});

using CURL_Async = void*; // system's state

CURL_Async CURL_async_create();
void CURL_async_destroy(CURL_Async curl_async);

where CURL_Async is the system itself, since user does not care what’s that exactly, it’s hidden under void*. User could create the system, use it and, once not needed, destroy - to clean up resources, if any.

void CURL_async_tick(CURL_Async curl_async);

This is the chance for a system to actually do some work over time and invoke user-provided callbacks, if needed.

Lastly, to give a user some controll over data in the callback, we pass opaque void* pointer around:

// main async callback API
void CURL_async_get(CURL_Async curl_async
    , const std::string& url
    , void* user_data
    , void (*callback)(void* user_data, std::string response));

user_data could be anything, system gives it back when invoking callback. This is user responsibility to ensure that pointer is valid all the time while request is in progress.

// libcurl bookkeeping
using CURL_Async = void*;
CURL_Async CURL_async_create();
void CURL_async_destroy(CURL_Async curl_async);
void CURL_async_tick(CURL_Async curl_async);

// main async callback API
void CURL_async_get(CURL_Async curl_async
    , const std::string& url
    , void* user_data
    , void (*callback)(void* user_data, std::string response));

4.2 # note on C-style API (vs C++)

For C-style API above, with C++, “the system” could be a class, callback could be std::function<> to accept anything, generally making it less verbose, having something like this:

// the API:
class CURL_Async
{
public:
    void get(const std::string& url, std::function<void (std::string)>);
    void tick();
};

// the use:
CURL_Async curl;
curl.get("localhost:5001/file1.txt", [](std::string r)
{
    std::println("{}", r);
});
curl.tick(); // etc

However, C-style API we have is defacto standard, familiar and reconized for asynchronous APIs with callbacks (citation needed).

The rest of asynchronous APIs implementations below are built on top of C-style callback API, as a basic building block to cover similar callbacks-based APIs.

4.3 # implementing with libcurl multi

using CURL_Async = void*;
CURL_Async CURL_async_create();
void CURL_async_destroy(CURL_Async curl_async);
void CURL_async_tick(CURL_Async curl_async);
void CURL_async_get(CURL_Async curl_async
    , const std::string& url
    , void* user_data
    , void (*callback)(void* user_data, std::string response));

internally, lets have CURL_AsyncScheduler class to handle adding requests, updating/ticking libcurl event loop and, in general, to represent our whole CURL_Async system state:

struct CURL_AsyncScheduler
{
    CURL_AsyncScheduler();
    ~CURL_AsyncScheduler();
    // no copy, no move
    CURL_AsyncScheduler(const CURL_AsyncScheduler&) = delete;

    using Callback = std::function<void (CURL* curl_easy)>;

    void tick();
    void add_request(CURL* curl_easy, Callback on_finish);

    // our state
    CURLM* _multi_curl = nullptr;
    std::unordered_map<CURL*, Callback> _curl_to_callback;
};

CURL_Async CURL_async_create()
{
    CURL_AsyncScheduler* scheduler = new(std::nothrow) CURL_AsyncScheduler();
    assert(scheduler);
    return scheduler;
}

void CURL_async_destroy(CURL_Async curl_async)
{
    assert(curl_async);
    CURL_AsyncScheduler* scheduler = static_cast<CURL_AsyncScheduler*>(curl_async);
    delete scheduler;
}

Done. Now, user just needs to pass CURL_Async handle around. Before implementing internals, lets have a helper function that gets actual CURL_AsyncScheduler instance from opaque handle:

CURL_AsyncScheduler& CURL_scheduler(CURL_Async curl_async)
{
    CURL_AsyncScheduler* scheduler = static_cast<CURL_AsyncScheduler*>(curl_async);
    assert(scheduler);
    return *scheduler;
}

It’s not exposed to the user in any way. Lets implement our main API in terms of our internal scheduler:

void CURL_async_get(CURL_Async curl_async
    , const std::string& url
    , void* user_data
    , void (*callback)(void* user_data, std::string response))
{
    // 1. setup curl easy handle
    CURL* curl_easy = curl_easy_init();
    assert(curl_easy);
    CURLcode status = curl_easy_setopt(curl_easy, CURLOPT_URL, url.c_str());
    assert(status == CURLE_OK);
    status = curl_easy_setopt(curl_easy, CURLOPT_FOLLOWLOCATION, 1L);
    assert(status == CURLE_OK);
    
    // 2. write response data to separate std::string
    std::string* state = new std::string{};
    status = curl_easy_setopt(curl_easy, CURLOPT_WRITEFUNCTION, CURL_OnWriteCallback);
    assert(status == CURLE_OK);
    status = curl_easy_setopt(curl_easy, CURLOPT_WRITEDATA, state);
    assert(status == CURLE_OK);

    // 3. associate with multi handle/event loop
    CURL_scheduler(curl_async).add_request(curl_easy
        , [state, user_data, callback](CURL* curl_easy)
    {
        long response_code = -1;
        const CURLcode status = curl_easy_getinfo(curl_easy, CURLINFO_RESPONSE_CODE, &response_code);
        assert(status == CURLE_OK);
        assert(response_code == 200L);
        curl_easy_cleanup(curl_easy);
        std::string data = std::move(*state);
        delete state;
        callback(user_data, std::move(data));
    });
}

It could be done another way around, eliminating the need for separate std::string allocation and few more optimizations, mainly with the help of associating user data with curl easy handle/CURLOPT_PRIVATE. However, it’s good enough for illustrative purposes.

CURL_AsyncScheduler::CURL_AsyncScheduler()
{
    const CURLcode status = curl_global_init(CURL_GLOBAL_ALL);
    assert(status == CURLE_OK);
    _multi_curl = curl_multi_init();
    assert(_multi_curl);
}

CURL_AsyncScheduler::~CURL_AsyncScheduler()
{
    const CURLMcode status = curl_multi_cleanup(_multi_curl);
    assert(status == CURLM_OK);
    curl_global_cleanup();
}

void CURL_AsyncScheduler::add_request(CURL* curl_easy, Callback on_finish)
{
    assert(on_finish);
    assert(curl_easy);
    assert(!_curl_to_callback.contains(curl_easy));

    const CURLMcode status = curl_multi_add_handle(_multi_curl, curl_easy);
    assert(status == CURLM_OK);
    _curl_to_callback[curl_easy] = std::move(on_finish);
}

_curl_to_callback map is used to be able to retrieve callback later, given curl easy handle (CURL*).

void CURL_async_tick(CURL_Async curl_async)
{
    CURL_scheduler(curl_async).tick();
}

void CURL_AsyncScheduler::tick()
{
    int running_handles = -1;
    CURLMcode status = curl_multi_perform(_multi_curl, &running_handles);
    assert(status == CURLM_OK);
    int msgs_in_queue = 0;
    while (CURLMsg* m = curl_multi_info_read(_multi_curl, &msgs_in_queue))
    {
        if (m->msg != CURLMSG_DONE)
        {
            continue;
        }
        CURL* curl_easy = m->easy_handle;
        assert(curl_easy);
        status = curl_multi_remove_handle(_multi_curl, curl_easy);
        assert(status == CURLM_OK);
        auto it = _curl_to_callback.find(curl_easy);
        assert(it != _curl_to_callback.end());
        Callback callback = std::move(it->second);
        assert(callback);
        (void)_curl_to_callback.erase(it);
        callback(curl_easy);
    }
}

The main part of event loop is the call to curl_multi_perform(). Once done we ask for easy handle requests that were completed, search for an associated callback for each request and invoke it.

Note, there are no threads involved and it’s possible to create many GET requests at once with multiple calls to CURL_async_get() - libcurl will manage them all together.

Again, it’s user responsibility to drive libcurl with a periodic calls to CURL_async_tick(). Lets do single request with the API above (source code):

#include <print>

int main()
{
    struct State
    {
        std::string response;
        bool done = false;
    };
    CURL_Async curl_async = CURL_async_create();
    State state;
    CURL_async_get(curl_async, "localhost:5001/file1.txt", &state
        , [](void* user_data, std::string response)
    {
        State& state = *static_cast<State*>(user_data);
        state.response = std::move(response);
        state.done = true;
    });
    while (!state.done)
    {
        CURL_async_tick(curl_async);
    }
    CURL_async_destroy(curl_async);

    std::println("async response: '{}'", state.response);
}

5 # blocking, synchronous (App_Blocking)

5.1 # on error handling

# assume success always (tooling)

# implicit, return empty string

# status code, out parameter (std::filesystem-style)

# optional

# exceptions

# result/variant-like

# result/tuple-like

# result/specialized

6 # async polling, tasks (App_Tasks)

7 # blocking std::future/promise

8 # async polling, std::future/promise

9 # async, callbacks (App_Callbacks)

10 # async, callbacks + polling (tasks, handle)

11 # async with statefull/implicit callback (state.on_X.subscribe/delegates)

12 # building C++20 coroutines API

const std::string response = co_await CURL_await_get(
    curl_async, "localhost:5001/file1.txt");
// use `response` as a usual variable, no callbacks

There are several moving and a bit unrelative parts to have working coroutines code. First, coroutine function return type needs to be built, just to be able to write any/empty coroutine:

Co_Task coro_work()
{
    co_return;
}

Next, there is a need to write coroutine awaitable to be able to co_await some work, specifically, GET request:

Co_Task coro_work(CURL_Async curl_async)
{
    std::string response = co_await CURL_await_get(curl_async
        , "localhost:5001/file1.txt");
    co_return;
}

And, finally, there are some challenges to have a code that has several GET requests on the fly with coroutines.

12.1 # C++ coroutines, basic task type

There is a trick of writing some basic C++20 coroutines code - listen to compiler. Lets see what it takes to make the next code “work”:

Co_Task coro_work()
{
    co_return;
}

struct Co_Task {};

Co_Task coro_work()
{
    co_return;
}

#include <coroutine>

struct Co_Task
{
    struct promise_type {};
};

Co_Task coro_work()
{
    co_return;
}

Ah, so promise_type should have get_return_object(), initial_suspend() and final_suspend() member functions. Return types are unclear, unfortunately. To speed-up things, we know that get_return_object() should return Co_Task. For initial_suspend() and final_suspend() we’ll go with std::suspend_always awaitables for now. That gives:

#include <coroutine>

struct Co_Task
{
    struct promise_type
    {
        Co_Task get_return_object()           { return {}; }
        std::suspend_always initial_suspend() { return {}; }
        std::suspend_always final_suspend()   { return {}; }
    };
};

Co_Task coro_work()
{
    co_return;
}

Since our coro_work() coroutine has just co_return, we should provide return_void() member function. With unhandled_exception(), we have:

struct promise_type
{
    Co_Task get_return_object()           { return {}; }
    std::suspend_always initial_suspend() { return {}; }
    std::suspend_always final_suspend()   { return {}; }
    void return_void()                    {}
    void unhandled_exception()            {}
};

#include <coroutine>

struct Co_Task
{
    struct promise_type
    {
        Co_Task get_return_object()                  { return {}; }
        std::suspend_always initial_suspend()        { return {}; }
        std::suspend_always final_suspend() noexcept { return {}; }
        void return_void()                           {}
        void unhandled_exception()                   {}
    };
};

Co_Task coro_work()
{
    co_return;
}

compiles! We just need to fill in details and implement given functions properly.

There are way too many different ways to implement coroutine task/promise types. There are no constraints and, in general, it all depends on your design and needs. We’ll go with owning coroutine task type:

For now, lets proceed with implementation. Since we own coroutine, our Co_Task needs to have destructor, should be move-only:

struct Co_Task
{
    struct promise_type;
    using co_handle = std::coroutine_handle<promise_type>;

    struct promise_type
    {
        Co_Task get_return_object()
        {
            return Co_Task{co_handle::from_promise(*this)};
        }
        // ...
    };

    Co_Task(co_handle coro)
        : _coro{coro} {}
    Co_Task(Co_Task&& rhs) noexcept
        : _coro{std::exchange(rhs._coro, {})} { }
    Co_Task(const Co_Task&) = delete;
    ~Co_Task() noexcept
    {
        if (_coro)
        {
            _coro.destroy();
        }
    }

    co_handle _coro;
};

In short, when we call coro_work(), compiler creates Co_Task::promise_type and invokes get_return_object() to be able to return an instance of Co_Task to the user. Here, in get_return_object() there is a way to get an access to std::coroutine_handle<> - the only way to interact with just alocated coroutine. Once Co_Task is created, we return it to the user. It’s up to the user to manage Co_Task. In our case, we own just created coroutine, hence if Co_Task is destroyed, we assume coroutine is in suspended state and destroy it too.

std::suspend_always promise_type::initial_suspend()
{
    return {};
}

std::suspend_always promise_type::final_suspend() noexcept
{
    return {};
}

void promise_type::return_void()
{
    // yeah, we return void. Nothing to do
}

void promise_type::unhandled_exception()
{
    // crash, no exceptions handling
    assert(false);
}

Co_Task coro_work()
{
    std::println("inside coro_work");
    co_return;
}

int main()
{
    Co_Task coro = coro_work(); 
}

which runs and… prints nothing since our coroutine is created and immediately suspended even before executing first print.

void Co_Task::resume()
{
    assert(_coro);
    assert(!_coro.done());
    _coro.resume();
}

Co_Task coro_work()
{
    std::println("inside coro_work");
    co_return;
}

int main()
{
    std::println("-- before coro_work()");
    Co_Task coro = coro_work();
    std::println("-- after coro_work()");
    coro.resume();
    std::println("-- after resume()");
}

12.2 # C++ coroutines, basic await

Given that we can have simplest coroutine, what does it take to co_await? Lets try to compile:

struct Co_CurlAsync {};

Co_Task coro_work()
{
    co_await Co_CurlAsync{};
    co_return;
}

So co_await requires “awaiter” to have those 3 functions. We can think about awaiter as something that:

The compiler asks awaiter, specifically, Co_CurlAsync with bool await_ready() if operation is done/ready or is in progress. If awaiter returns false, the compiler switches current coroutine state to “suspended” and invokes awaiter’s await_suspend(std::coroutine_handle<> coro) customization point which allows to remember current coroutine coro handle that goes to suspend state, to call .resume() later, once operation is done. Once coroutine is resumed, compiler asks for a value from last awaiter responsible for suspend.

In short, we can have Co_CurlAsync awaiter that tells that (1) operation is not ready yet (2) on suspend, resumes coroutine immediately and (3) returns nothing:

struct Co_CurlAsync
{
    bool await_ready()
    {
        return false;
    }

    void await_suspend(std::coroutine_handle<> coro)
    {
        std::println("-- inside suspend, resuming immediately");
        coro.resume();
    }

    void await_resume()
    {
        std::println("-- resume");
    }
};

Co_Task coro_work()
{
    std::println("before co_await");
    co_await Co_CurlAsync{};
    std::println("after co_await");
    co_return;
}

int main()
{
    Co_Task coro = coro_work();
    coro.resume();
}

Now, on suspend, we did nothing, but immediately resumed coroutine. But we also could start an async operation and, on finish, resume the coroutine.

12.3 # C++ coroutines, await callback with a crash

struct Co_CurlAsync
{
    CURL_Async _curl_async{};
    std::string _url;
    std::coroutine_handle<> _coro;
    std::string _response;

    bool await_ready()
    { // 1. CURL_async_get() is not yet started, force coroutine suspend:
        return false;
    }

    void await_suspend(std::coroutine_handle<> coro)
    { // 2. remember coroutine handle, start request, resume on finish:
        _coro = coro;

        CURL_async_get(_curl_async, _url, this
            , [](void* user_data, std::string response)
        {
            Co_CurlAsync& self = *static_cast<Co_CurlAsync*>(user_data);
            self._response = std::move(response);
            self._coro.resume();
        });
    }

    std::string await_resume()
    { // 3. after resume, return response:
        return std::move(_response);
    }
};

Co_CurlAsync CURL_await_get(CURL_Async curl_async, const std::string& url)
{
    return Co_CurlAsync{._curl_async = curl_async, ._url = url};
}

So, now co_await CURL_await_get(..., "url") should compile and kind-a work. As always, there are few moving part.

When coroutine function (represented as std::coroutine_handle<>) co_awaits our CURL awaiter - Co_CurlAsync, we:

There is one big issue there: what if we start a request with CURL_async_get(), coroutine suspends, BUT user discards Co_Task value that destroys coroutine, making std::coroutine_handle<> we remembered - dangling? There are several possible solutions, but lets see the current code in action by writing our main() function:

Co_Task coro_main(CURL_Async curl_async)
{
    const std::string response = co_await CURL_await_get(
        curl_async, "localhost:5001/file1.txt");

    std::println("coro_main response: '{}'", response);
    co_return;
}

int main()
{
    CURL_Async curl_async = CURL_async_create();
    Co_Task task = coro_main(curl_async);
    task.resume();
    while (task.is_in_progress())
    {
        CURL_async_tick(curl_async);
    }
    CURL_async_destroy(curl_async);
}

Here, we setup CURL_Async, as usual, and drive the loop until coroutine is in progress:

bool Co_Task::is_in_progress() const
{
    assert(_coro);
    return !_coro.done();
}

However, that works because we wait for coroutine until full complete. If we discard Co_Task too early, there is going to be a crash:

int main()
{
    CURL_Async curl_async = CURL_async_create();

    {
        Co_Task task = coro_main(curl_async);
        task.resume(); // run
    }   // **destroy**

    while (true)
    {
        CURL_async_tick(curl_async); // resume coroutine from there
    }
    CURL_async_destroy(curl_async);
}

In short, we start request, then .destroy() coroutine then try to resume dangling coroutine inside a callback with a call to .resume() even using stale pointer to awaiter (user data in the callback).

1st solution could be the best but changes completely the semantics of Co_Task, does not allow to easily have Co_Task<T> that return some value and requires to be able to change Co_Task internals.

3rd solution requires changes to our basic C-style callback API which we assume we can’t do.

4th solution is the most ineficient and requires no changes neither in Co_Task nor in callback API.