Adventures with \Device\Afd - a simple server

The only difference between a client and a server is the way in which the connection is established. For a client, you create a socket and call “connect” on it. For a server, we have a socket that is “listening” for connections “accepts” new incoming connections and returns a socket that is then indistinguishable from a client connection.

In the past, I’ve created bad abstractions at this point. A socket connection and a listening socket are both represented by the operating system API as the same type, and the only differences are the calls that you make on the type. At a higher level of abstraction, they’re completely different.

In my exploration of the \Device\Afd interface I’ve decided to separate the types that I create to represent “listening” and “connection” sockets.

Last time, I put together a simple client. This time I’m going to explore what’s needed to create a simple server. The first thing is a new type, a tcp_listening_socket that only exposes the connection accepting interface of the underlying socket and only deals with the events that these API calls generate in the AFD system.

A simple, single-threaded, server

The aim here is to end up with a simple server that is very similar in structure to our simple client.

Something like this, where the only difference between the code in our client is that we create a server object rather than a client object and we can listen() rather than connect().

int main(int argc, char **argv)
{
   InitialiseWinsock();

   try
   {
      const auto handles = CreateAfdAndIOCP();

      multi_connection_afd_system afd(handles.afd);

      afd_handle handle(afd, 0);

      echo_server server(handle);

      sockaddr_in address{};

      address.sin_family = AF_INET;
      address.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
      address.sin_port = htons(5050);

      const int backlog = 10;

      server.listen(reinterpret_cast<const sockaddr &>(address), sizeof address, backlog);

      while (!server.done())
      {
         // process events

         auto *pAfd = GetCompletionAs<afd_system_events>(handles.iocp, INFINITE);

         if (pAfd)
         {
            pAfd->handle_events();
         }
         else
         {
            throw std::exception("failed to process events");
         }
      }
   }
   catch (std::exception &e)
   {
      std::cout << "exception: " << e.what() << std::endl;
   }

   std::cout << "all done" << std::endl;

   return 0;
}

A listening socket

Our server needs to be notified when a new connection is available to accept. This is the main event that our listening socket needs to support.

The server will then deal with this event as follows:

void on_incoming_connections(
   tcp_listening_socket &s) override
{
   std::cout << "on_incoming_connections" << std::endl;

   bool accepting = true;

   while (accepting)
   {
      sockaddr_in client_address{};

      int client_address_length = sizeof client_address;

      SOCKET client_socket = s.accept(reinterpret_cast<sockaddr &>(client_address), client_address_length);

      accepting = (client_socket != INVALID_SOCKET);

      if (accepting)
      {

Essentially, when we are told that new connections are available we should attempt to accept these new connections until there are no more to accept. In the code above, each time through the loop, we end up with a client_socket and this could be wrapped in our tcp_socket class and used much as we use it in the simple client.

The tcp_listening_socket is very similar in structure to our tcp_socket but only exposes the bind(), listen() and accept() and close() functions. The important part, is listen() which puts the socket into listening mode and issues the first poll to the associated AFD system to wait for events.

void tcp_listening_socket::listen(
   const int backlog)
{
   if (SOCKET_ERROR == ::listen(s, backlog))
   {
      throw std::exception("failed to listen");
   }

   connection_state = state::listening;

   events = AllEvents;
   
   afd.poll(events);
}

Whilst we poll for AllEvents we actually only need to poll for AFD_POLL_ACCEPT and AFD_POLL_ABORT.

In our event handler we call the appropriate callbacks when the events occur and, in a change to how we deal with event handling in our tcp_socket, the last thing we do is poll for more events. We do this as the listening state is a continuous state so we should continue to poll until the socket is closed.

ULONG tcp_listening_socket::handle_events(
   const ULONG eventsToHandle,
   const NTSTATUS status)
{
   (void)status;

   if (connection_state == state::listening)
   {
      if (AFD_POLL_ACCEPT & eventsToHandle)
      {
         callbacks.on_incoming_connections(*this);
      }
   }

   if (AFD_POLL_ABORT & eventsToHandle)
   {
      connection_state = state::disconnected;

      callbacks.on_connection_reset(*this);

      events = 0;
   }

   if (AFD_POLL_LOCAL_CLOSE & eventsToHandle)
   {
      connection_state = state::disconnected;

      callbacks.on_disconnected(*this);

      events = 0;
   }

   if(events)
   {
      afd.poll(events);
   }

   return events;
}

Dealing with multiple sockets with AFD

And now we have to deal with the problem that has been lurking in the code for a while now. The fact that we can currently only deal with a single socket with our afd_system object.

The current simple server code accepts connections like this:

SOCKET client_socket = s.accept(reinterpret_cast<sockaddr &>(client_address), client_address_length);

accepting = (client_socket != INVALID_SOCKET);

if (accepting)
{
   std::cout << "new connection accepted" << std::endl;

   static const char *pMessage = "TODO\r\n";

   ::send(client_socket, pMessage, 6, 0);

   // this is where we need to support multiple sockets in the afd system...

   ::shutdown(client_socket, SD_SEND);
   ::closesocket(client_socket);
}

Ideally we would create a tcp_socket, attach it to the client_socket, associate it with the afd_system and allow it to continue in an event-driven manner, but we can’t as the afd_system only supports a single socket and that socket is the listening socket.

So, the next step is to go back to our code that interacts with the \Device\Afd interface and add support for multiple sockets.

Wrapping up

Building a listening_socket and a simple server was fairly easy as it was just a case of handling one new kind of event. Now we can avoid the complicated code no longer, and we have to dive into dealing with multiple connections.

More on AFD

• Adventures with \\Device\\Afd
• Test Driven Understanding
• Test Driven Design
• A simple client
• A simple server - this post