01 Jan 2013

#Low level socket programming in POSIX and Core Foundation

##Simple CFSockets without byte-streams

This post describes setting up a basic socket communication. Generally speaking, the best practice is to use the highest level APIs for network connections. This post is specifically for understanding the CFSocket API at its lowest level for the purpose of if you are writing a cross-platform application, a high-performance demon or service, or to support protocols other than TCP. These things can not be done with the higher level APIs. For example, to listen to incoming network connections, we must either use CFSocket or POSIX sockets.

POSIX sockets are a very basic level of communication over a network. A socket is an address associated with a port number. Once we have a socket, we can connect to it to receive and send data. CFSocket operates over POSIX sockets, providing an abstraction that can be integrated into a run loop. It has a few improvements and convenience functions over using POSIX sockets.

Besides run loop integration, there are a few other reasons we will be using Core Foundation over the POSIX API. In the POSIX API we must handle differences in protocols ourselves. For example, we must explicitly handle if we are using IPv4 verses IPv6 addresses. If we are running our code on an Apple mobile device, the cellular radio and on-demand VPN may turn off to save the battery. POSIX calls will not automatically wake these up, which is why it’s generally advised to avoid POSIX networking in this environment.

There are basically two main ways to use CFSocket. The first is for simple connections where there is no byte stream such as datagram or listening sockets. This tutorial addresses this. To use sockets for byte-stream TCP connections, the CFStream API has ‘socket additions’, which this reference should be used. If you need to operate at the socket level but are sending and receiving large amounts of data, it is highly recommended to use the socket additions for CFStream. Streams can also be connected to a hostname without having to first manually resolve the address. They more closely resemble the communication system of other higher level APIs, where there is a read and write stream and the Core Foundation streams can be toll-free bridged to their Cocoa counterparts. CFStream has already been covered in this tutorial.

At this non-stream level, CFSocket lets us connect to a socket to send raw data. In this way we have explicit control over the communication. However, it means we must also handle the fact that sending and receiving data will block whatever thread we are currently operating in. To avoid this we can use run loop integration; we can listen to specific events such as when the socket buffer is ready to have data sent to it, and when data has been received by the kernel and is ready to be processed. Since there are hardly any tutorials at this level of handling reading and writing ourselves, we will set up a simple example. You may want to use this functionality to connect directly to a remote machine and implement your own communication synchronization. But to make this post accessible to everyone, we will connect to a SMTP server as an example. (Note that SMTP can be used over TCP, UDP or a custom protocol as it is a protocol that is independent of the transmission system). To get started, we need to include some networking headers and set up a global run loop source variable.

#include <arpa/inet.h> //for PF_INET, SOCK_STREAM, IPPROTO_TCP etc
#include <netdb.h> //for gethostbyname

CFRunLoopSourceRef gSocketSource = NULL;

To connect to a socket at this level, the first step is to resolve any domain name to an ip address. The POSIX layer is a connect-by-ip API, not by domain name. Lets say we have been given a domain name in which a DNS server would need to resolve to an ip address. This ip address may change so we must not hard code a resolved ip address. The POSIX way of resolving this would be to use the gethostbyname() function like so:

struct sockaddr_in socketAddress;
memset(&socketAddress, 0, sizeof(socketAddress));
socketAddress.sin_len = sizeof(socketAddress);
socketAddress.sin_family = AF_INET;
socketAddress.sin_port = htons(25);
const char *c = "";
struct hostent *host_entry = gethostbyname(c);
if (host_entry)
    //and continue to connect to socket…
    char *ip_addr = inet_ntoa(*((struct in_addr *)host_entry->h_addr_list[0]));
    socketAddress.sin_addr.s_addr = inet_addr(ip_addr);

    CFDataRef socketData = CFDataCreate(kCFAllocatorDefault, (const UInt8 *)&socketAddress, sizeof(socketAddress));
    CFSocketConnectToAddress(theSocket, socketData, 30);

Note that for the sockadd_in structure, we set the .sin_family to AF_INET for the IP version 4 address family. For the .sin_port we wrap the port number around the host-to-network htons() function. Network protocols require network byte order, which is big-endian. This means that any little-endian machines need to swap the order of bytes, while on big-endian machines this function will do nothing. We also set the length of this structure if we are porting our code, as some system such as ones running BSD 4.4 need this to be set. That’s a lot of stuff to do to resolve and connect to a socket. Additionally, gethostbyname will call a DNS function (such as res_query on Mac OS/iOS) and if the address is not already in the configuration (/etc/host.conf for example) this function will block the thread. One solution would to use pthread_t to spawn a thread. However, since our goal is to work with run loops, we can instead use Core Foundation’s resolution service to do exactly that. Lets create a function to start the process

void ConnectSocketAndSendCommandArray(CFArrayRef array)


Here is the updated code in place of the POSIX code:

void ConnectSocketAndSendCommandArray(CFArrayRef array)
	CFHostRef host = CFHostCreateWithName(kCFAllocatorDefault, CFSTR(""));
    CFStreamError streamError;
    CFHostClientContext hostContext = {0, (void *)array, RetainSocketStreamHandle, ReleaseSocketStreamHandle, NULL};
    CFHostSetClient(host, ResolutionCallBackFunction, &hostContext);
    CFHostScheduleWithRunLoop(host, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);
    Boolean started = CFHostStartInfoResolution(host, kCFHostAddresses, &streamError);
    if (!started)
        printf("Could not start info resolution\n");

First we create a host object, CFHostRef, with the name we want to resolve. We set up a context -CFHostClientContext as we will be passing a CFArray object that includes data to send to the server. The context also includes retain and release functions to make sure the memory is managed properly. If this is unfamiliar ground, see the Retain and Release callback functions section of the CFNetwork POST and callback functions tutorial. In order to schedule this operation on the run loop so that it will not block the main thread, we use the CFHostScheduleWithRunLoop() function. Finally, we call CFHostStartInfoResolution() to start the DNS query. Here are the retain and release call back functions.

const void *RetainSocketStreamHandle(const void *info)
    return info;

void ReleaseSocketStreamHandle(const void *info)
    if (info)

And the callback function prototype looks like this

void ResolutionCallBackFunction(CFHostRef host, CFHostInfoType typeInfo, const CFStreamError *error, void *info);

Now lets add content to this callback function. If resolution is successful, the call back will be called and the CFHost object is passed into this function. At this point we can see if there is an array of resolved IPs by calling CFHostGetAddressing(). This function can return multiple IPs if, for example, a machine has several network interfaces such as a WIFI and Ethernet card. At the application layer, we can not determine which IP might be ideal to connect to. Higher level APIs will attempt to determine which IP is the best to use but at this lower level that would need to be implemented ourselves. We take the first address object in the array, which will be a CFDataRef. The CFData object just holds a sockadd_in structure which means we can access this structure directly by using CFDataGetBytePtr(). In this example we first copy the data using CFDataCreateCopy because we will make changes to the structure; we will change the port number to port 25 for SMTP. Once we set up the structure as we want it, we can simply connect to the socket by using CFSocketConnectToAddress().

NOTE: We must always close any connections that use the TCP protocol. Because this is a reliable protocol, it will remain open until an explicit close or a timeout occurs.

We will also need to create our socket object. This is pretty much the same idea as creating a POSIX socket with the socket() and bind() calls.

CFSocketContext context = {0, (void *)info, RetainSocketStreamHandle, ReleaseSocketStreamHandle, NULL};
    CFSocketRef theSocket = CFSocketCreate(kCFAllocatorDefault, PF_INET, SOCK_STREAM, IPPROTO_TCP, kCFSocketConnectCallBack, (CFSocketCallBack)SocketCallBack, &context);
    CFSocketSetSocketFlags(theSocket, kCFSocketCloseOnInvalidate);

Here we pass in PF_INET for the IP version 4 “protocol family” (yes, that’s what the PF stands for). To have transport layer support, we use SOCK_STREAM as the socket type. TCP is a connection-oriented reliable protocol for a “stream” of octets, which is why it is called SOCK_STREAM. Other options are datagram - SOCK_DGRAM for UDP or SOCK_RAW which would be a raw socket interface without any transport layer support. Then we specify the call back events we want to listen for and what function will be called when any of those events occur. Specifying kCFSocketConnectCallBack, for example, means that the connection will be established in the background and once the connection is accepted, our callback function will then be called on the calling thread (our main thread in this case).

The CFSocketSetSocketFlags() function controls if specific callbacks are automatically reenabled after they fire, and if the POSIX native socket gets closed when the socket is invalidated with the CFSocketInvalidate() function. CFSocket Flags are

enum {
   kCFSocketAutomaticallyReenableReadCallBack = 1,
   kCFSocketAutomaticallyReenableAcceptCallBack = 2,
   kCFSocketAutomaticallyReenableDataCallBack = 3,
   kCFSocketAutomaticallyReenableWriteCallBack = 8,
   kCFSocketLeaveErrors = 64,
   kCFSocketCloseOnInvalidate = 128

We get more into the callbacks below. We do not want callbacks being fired repeatedly; interrupting the main thread so we do not set any of the callbacks to be automatically re-enabled.

Finally, since we do not want to block the thread until the connection is accepted, we schedule this socket on the run loop

gSocketSource = CFSocketCreateRunLoopSource(kCFAllocatorDefault, theSocket, 0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), gSocketSource, kCFRunLoopDefaultMode);

Here is the full callback code

void ResolutionCallBackFunction(CFHostRef host, CFHostInfoType typeInfo, const CFStreamError *error, void *info)
    Boolean hostsResolved;
    CFArrayRef addressesArray = CFHostGetAddressing(host, &hostsResolved);
    CFSocketContext context = {0, (void *)info, RetainSocketStreamHandle, ReleaseSocketStreamHandle, NULL};
    CFSocketRef theSocket = CFSocketCreate(kCFAllocatorDefault, PF_INET, SOCK_STREAM, IPPROTO_TCP, kCFSocketConnectCallBack, (CFSocketCallBack)SocketCallBack, &context);
    CFSocketSetSocketFlags(theSocket, kCFSocketCloseOnInvalidate);

    if (addressesArray && CFArrayGetCount(addressesArray))
        CFDataRef socketData = (CFDataRef)CFArrayGetValueAtIndex(addressesArray, 0);
        //Here our connection only accepts port 25 - so we must explicitly change this
        //first, we copy the data so we can change it
        CFDataRef socketDataCopy = CFDataCreateCopy(kCFAllocatorDefault, socketData);
        struct sockaddr_in *addressStruct = (struct sockaddr_in *)CFDataGetBytePtr(socketDataCopy);
        addressStruct->sin_port = htons(25);
        CFSocketError socketError = CFSocketConnectToAddress(theSocket, socketDataCopy, 30);
        if (socketError != kCFSocketSuccess)
            printf("Error sending login fail to socket connection\n");
    if (host)
    gSocketSource = CFSocketCreateRunLoopSource(kCFAllocatorDefault, theSocket, 0);
    CFRunLoopAddSource(CFRunLoopGetCurrent(), gSocketSource, kCFRunLoopDefaultMode);

Once connected, the OS will call back to our “SocketCallBack” function, in which case we can output any data received in the buffer using the recv() POSIX function. For example

void SocketCallBack(CFSocketRef socket, CFSocketCallBackType type, CFDataRef address, const void *data, void *info)
	UInt8 buffer[1024];
    bzero(buffer, sizeof(buffer));
    CFSocketNativeHandle sock = CFSocketGetNative(socket); // The native socket, used for BSD tasks
    //start receiving data
    recv(sock, buffer, sizeof(buffer), 0);
    printf("Output: %s\n", buffer);

For using native POSIX functions, we can get the native socket handle by calling the CFSocketGetNative() function. recv() stores the received data into the buffer which we then print to console. We can also send data to the buffer using the send() POSIX call, but first we should set a timeout by using the ‘timeval’ structure. We can set the seconds of this structure with .tv_sec. Note that .tv_usec should be initialized to zero if we are not using it as on some systems not setting it causes undefined behaviour

//timeout of 10 seconds
struct timeval tv;
tv.tv_sec = 10;
tv.tv_usec = 0;
setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));

char *sendChar = "HELO\n";
send(sock, sendChar, strlen(sendChar), 0);

In Core Foundation, a convenience function was created so that we can just pass in a timeout value instead, as well as pass a CFDataRef object to wrap the data we want to send. Here is the updated send code:

char *toSendChar = "HELO\n";
CFDataRef sendData = CFDataCreate(kCFAllocatorDefault, (const UInt8 *)toSendChar, strlen(toSendChar));
CFSocketSendData(socket, address, sendData, 30);

The next lines of code would then be to call recv() again to check what the server responded. The matter which method we use to send data, the send function will send data to a socket buffer that gets scheduled to be sent. However, if the socket buffer is full and can not receive any more messages, the function will block the thread. We should never make synchronous network calls on the main thread. send() and CFSocketSendData() are synchronous blocking calls that only return if the send succeeds as far as the kernel is aware, or if an error has occurred. (Remember, this is not the same as knowing if the data was received on the other end. This is implemented separately at the transport layer). recv() also blocks the thread until data has accumulated in the buffer to be received. Calling these alone would mean they should be on a separate thread. Again at this point, inside our connect callback function we could use the pthreads API to spawn a thread and add the send() and recv() functions there. This will work for simple read and write communications. But another option here is to use an asynchronous read and write model integrated into the run loop which we have been doing so far. Lets go back to the CFSocketCreate() function and add data read and write events that will also fire our callback function.

        CFSocketRef theSocket = CFSocketCreate(kCFAllocatorDefault, PF_INET, SOCK_STREAM, IPPROTO_TCP, kCFSocketConnectCallBack | kCFSocketDataCallBack | kCFSocketWriteCallBack, (CFSocketCallBack)SocketCallBack, &context);

Here we can determine when the socket is ready for write operations (kCFSocketWriteCallBack), and when data has arrived in the background and is ready to be received (kCFSocketDataCallBack).

Note that there is also a kCFSocketReadCallBack, which is different than kCFSocketDataCallBack. kCFSocketReadCallBack signifies that the socket buffer is ready to be read and we must then call revc(). The kCFSocketDataCallBack has already received the data in the background and passes it back as the “data” pointer in the call back function.

A gotcha about the write callback is that although we can receive and read data asynchronously in the background, with the kCFSocketWriteCallBack call back, we can check when the socket is ready to send data, but the send functions themselves are still blocking. So, if we are sending large amounts of data, we should either spawn a thread or use the CFStream API which uses read and write streams scheduled on the run loop in the same way we would implement HTTP message streams.

NOTE: As a final option, at the file control level we can set the POSIX send() function to non blocking I/O mode. The way to make a socket non-blocking in POSIX is to call:

#include <sys/fcntl.h>
fcntl(socketFileDescriptor, F_SETFL,  O_NONBLOCK);

Last but not least, remember to close the stream. The POSIX native call is close(). The CFSocketInvalidate() convenience function closes the socket, prevents the CFSocket from sending or receiving any more messages, as well as invalidates the run loop (if a specific source was created for the socket). Additionally, if the CFSocketContext has a release call back specified, the release function will be called at this time; releasing the ‘info’ data field.

The only time this function will not close the socket is if the kCFSocketCloseOnInvalidate flag was unset with the CFSocketSetSocketFlags function. In this case, a close() POSIX call is still needed.

**Closing the socket with either function does not release the socket object. You still need to explicitly call a CFRelease on the socket. **

Here is the cleanup code, added as a separate function

void CloseSocket(CFSocketRef socket)
    printf("Closing socket\n");
    //cleanup - invalidate below will also remove from runloop...
    if (gSocketSource)
        CFRunLoopRef currentRunLoop = CFRunLoopGetCurrent();
        if (CFRunLoopContainsSource(currentRunLoop, gSocketSource, kCFRunLoopDefaultMode))
            CFRunLoopRemoveSource(currentRunLoop, gSocketSource, kCFRunLoopDefaultMode);
    if (socket) //close socket
        if (CFSocketIsValid(socket))

SMTP Example:

Now that we have a simple example of a socket connection and since we are not using CFStream’s socket additions, the last part of this post takes the idea further to provide synchronization between reading and writing. Since we are connecting to a SMTP server, the problem arises that we must send data in a particular order, and only after receiving the correct response from the server. Keep in mind, the run loop integration calls back when the socket is ready to be written to and when data has been received and is ready to be processed. When the socket is ready to be written to, we need to know what the next SMTP command should be to send to the socket. When the socket has received data, we need to know what the correct response code should be. To make things simple, we will have 3 commands to send and receive responses to. A connection, a SMTP HELO command, and a SMTP QUIT command. This is a contrived example but the idea can be used if you are sending and parsing information with a particular order in a custom setup. We can start adding code to our main application start-up function (main or didFinishLaunchingWithOptions on Apple platforms). SMTP response codes for these 3 steps are as follows

//SMTP codes
//220 service is running
//250 action OK
//221 closing connection

The basic idea would be to iterate over an array of commands to send to the server. Each object in the array would be a dictionary containing a sequence or step number, the command to send, and the correct response that should be received. This way we could add error control to close the connection, or other action, if the correct response is not received. While the index is not mandatory, we can even setup an enum for readability


CFIndex currentStep = kSMTPStepConnect;

Lets setup a CFArray containing 3 steps; 3 CFDictionary objects. The dictionary will contain a CFNumber for the step, a CFString for the command, and a CFString for the result code. CFNullRefs are used if there is no command to send, as in the first connection step. You can package a NULL into a container object with the kCFNull singleton instance.

CFDictionaryRef theDict[3];
//keys for the dictionary
CFStringRef keys[3] = {CFSTR("keyStep"), CFSTR("keyCommand"), CFSTR("keyResult")};

//first step
CFIndex stepIndex = 0;
CFNumberRef number = CFNumberCreate(kCFAllocatorDefault, kCFNumberCFIndexType, &stepIndex);
const void *firstValues[3] = {number, kCFNull, CFSTR("220")};
theDict[0] = CFDictionaryCreate(kCFAllocatorDefault, (const void **)keys, firstValues, 3, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);

//second step
stepIndex = 1;
number = CFNumberCreate(kCFAllocatorDefault, kCFNumberCFIndexType, &stepIndex);
const void *secondValues[3] = {number, CFSTR("HELO\n"), CFSTR("250")};
theDict[1] = CFDictionaryCreate(kCFAllocatorDefault, (const void **)keys, secondValues, 3, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);

//third step
stepIndex = 2;
number = CFNumberCreate(kCFAllocatorDefault, kCFNumberCFIndexType, &stepIndex);
const void *thirdValues[3] = {number, CFSTR("QUIT\n"), CFSTR("221")};
theDict[2] = CFDictionaryCreate(kCFAllocatorDefault, (const void **)keys, thirdValues, 3, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);

//add to array
CFArrayRef stepArray = CFArrayCreate(kCFAllocatorDefault, (const void **)theDict, 3, &kCFTypeArrayCallBacks);

Now that we have an array, we can pass it to our function that starts the socket connection. Add this to the bottom of the above code


Our “ResolutionCallBackFunction” remains unchanged. The “SocketCallBack” function is the one that has the most changes. What we will do, is set up flow control based on the CFSocketCallBackType, especially to differentiate between kCFSocketDataCallBack and kCFSocketWriteCallBack events. In the data received callback section, we will check if we have the correct server result number for the current step. If we do, everything is okay and we can iterate to the next step.

One problem that would arise is that the call backs will continuously fire so long as the socket is writable, or if data continues to arrive. We would like to have the write callback fire only when we want to write, that is, when we have received the correct result from the server and would like to continue by writing the next command to the server. While writing, we will also ignore the data callback. In order to do this we use the CFSocketEnableCallBacks() and CFSocketDisableCallBacks() functions. Simply passing the callback we want into one of these functions will either temporality turn it on or off. For example, CFSocketDisableCallBacks(socket, kCFSocketWriteCallBack) will turn off the write call back from firing. Here is the full code for the SocketCallBack function

void SocketCallBack(CFSocketRef socket, CFSocketCallBackType type, CFDataRef address, const void *data, void *info)
    CFArrayRef stepArray = (CFArrayRef)info;
    CFDictionaryRef currentStepDictionary = NULL;
    if (currentStep < CFArrayGetCount(stepArray))
        currentStepDictionary = (CFDictionaryRef)CFArrayGetValueAtIndex(stepArray, currentStep);
    if (!currentStepDictionary)
    if (type == kCFSocketConnectCallBack)
    else if (type == kCFSocketDataCallBack)
        printf("buffer has read data\n");
        UInt8 *buffer = (UInt8 *)CFDataGetBytePtr((CFDataRef)data);
        CFIndex length = CFDataGetLength((CFDataRef)data);
        CFStringRef returnedString = CFStringCreateWithBytes(kCFAllocatorDefault, buffer, length, kCFStringEncodingUTF8, TRUE);

        //if we have correct response code
        if ( CFStringFind(returnedString, (CFStringRef)CFDictionaryGetValue(currentStepDictionary, CFSTR("keyResult")), 0).location != kCFNotFound )
            //move on to next step

            //turn off read data and enable write
            CFSocketDisableCallBacks(socket, kCFSocketDataCallBack);
            CFSocketEnableCallBacks(socket, kCFSocketWriteCallBack);

        //if finished, close the socket
        CFNumberRef theStepNumber = (CFNumberRef)CFDictionaryGetValue(currentStepDictionary, CFSTR("keyStep"));
        CFIndex step;
        CFNumberGetValue(theStepNumber, kCFNumberCFIndexType, &step);
        if (step == kSMTPStepClose)
    else if (type == kCFSocketWriteCallBack)
        printf("Buffer Writable\n");
        CFNumberRef theStepNumber = (CFNumberRef)CFDictionaryGetValue(currentStepDictionary, CFSTR("keyStep"));
        CFIndex step;
        CFNumberGetValue(theStepNumber, kCFNumberCFIndexType, &step);
        if (step == currentStep)
            CFStringRef string = (CFStringRef)CFDictionaryGetValue(currentStepDictionary, CFSTR("keyCommand"));
            if (string && ( CFGetTypeID(string) != CFNullGetTypeID() ))
                //turn off write, enable read data
                CFSocketDisableCallBacks(socket, kCFSocketWriteCallBack);
                CFSocketEnableCallBacks(socket, kCFSocketDataCallBack);
                const char *sendChar = CFStringGetCStringPtr(string, kCFStringEncodingUTF8);
                printf("Writing %s\n", sendChar);
                CFDataRef sendData = CFDataCreate(kCFAllocatorDefault, (const UInt8 *)sendChar, strlen(sendChar));
                CFSocketSendData(socket, address, sendData, 30);
            } //end if string

        } //end if (step == currentStep)
    } //end if (type == kCFSocketWriteCallBack)

Here we check if we are on the right step in order to proceed. Print lines have been added so we can observe the flow. Additionally, the cleanup code we wrote earlier is used when we have reached the last step during the data read callback event. So there we have it! Again, this is example skeleton code, so it is missing error control. CFSocket is a unique library, in which we can operate at such a low level to have fine grained control. Remember that if you are using more standard TCP stream-based behaviour to try out the socket additions for CFStream. This concludes the post, and as always, the full source code can be downloaded here.