The RIO package provides an I/O layer on top of sockets that provides two core pieces of functionality: handling short reads and buffering of read data. If you don't use the RIO package, you will have to implement how to handle short reads yourselves for a correct implementation. Implementing buffering, though commonly done as a performance optimization that reduces the number of read() system calls, is not necessary for correctness.
Run it under strace. You may wish to set '-s 1024' to be able to examine all data written/read.
A simple trick is to use 'curl -v' and specify the same URL twice, as in:
$ curl -v http://cs3214.cs.vt.edu:9011/loadavg http://cs3214.cs.vt.edu:9011/loadavg * About to connect() to cs3214.cs.vt.edu port 9011 * Trying 128.173.41.123... connected * Connected to cs3214.cs.vt.edu (128.173.41.123) port 9011 ... Connection #0 to host cs3214.cs.vt.edu left intact * Re-using existing connection! (#0) with host cs3214.cs.vt.edu ... Connection #0 to host cs3214.cs.vt.edu left intact * Closing connection #0
Look for the lines shown above.
Your server may fail when executing the tests of the test harness in sequence.
One way to debug such problems is by using wrappers for valgrind and strace. To do this,
create files swrapper and vwrapper like so:
:::::::::::::: swrapper :::::::::::::: #!/bin/sh # # invoke strace, passing along any arguments strace -s 1024 -e network,read,write -ff -o stracelog ./sysstatd $* :::::::::::::: vwrapper :::::::::::::: #!/bin/sh # # invoke valgrind, passing along any arguments valgrind ./sysstatd $*
Make sure to make those scripts executable (chmod +x ?wrapper), then
you can run the test harness via server_unit_test.py -s ./vwrapper -o output
to see valgrind output in 'output' (respectively for swrapper).
No. CRLF is used only to separate header lines (and to end the header). The body consists of arbitrary content, which isn't necessarily line-oriented at all.
A possible reason is that you append additional bytes beyond the number you announced in Content-Length. curl skips whitespace when attempting to read a response. If the additional bytes you send are whitespace (such as \r\n), then curl will hide the problem.
To meet the basic requirements of the assignment, no. The only possible use I could see is if you wanted to implement a time-out feature without issuing a blocking read() call. In this case, you would call select() before read(), giving a read set that consists only of the file descriptor you're intending to read from. If select() times out without the file descriptor having become readable, you treat it as timeout. Otherwise, you perform a single read() call on the file descriptor which you know won't block.
An alternative to that is the use of a timer and a signal, as shown here; although select() is likely the simpler solution.
No. TCP uses a quadruple (src ip, src port, dst ip, dst port) to identify each connection, so different connections can go to the same dst port as long as the src ip or src port are different. You will obtain a socket that refers to a specific connection (i.e., client with a specific src ip/src port pair) as the return value from the accept() call.
No. You don't need to (and must not) call anything. After connecting, simply send your prefix, then start handling HTTP/1.1 requests on that same socket.
A TCP connection is bidirectional, and subsequent to the handshake that establishes a connection, the TCP protocol has in fact no memory of who established it (the TCP client) and who accepted it (the TCP server). It's like picking up a phone (or calling someone back on a phone) - once the phone call is in progress, both parties can speak and assume whichever roles are suited to their conversation.
As discussed in question 9, you don't call bind() at all. When you connect() to the relay server, the OS assigns a port to your connection on the machine from which you're connecting. (It does this for all TCP clients that don't call bind() - the common case.) As the connection is established, the NAT gateway eavesdrops on the connection and records this port in a map alongside the (random) port it assigned for the outside to use. When the outside peer (in our case, the relay server) replies, it'll rewrite (translate) the packet and reinsert the original port number before forwarding the packet so that the so-translated packet can be dispatched to the proper connection when it arrives back at the machine where your server runs.
The use of the 'Connection: close' header to announce that no more or requests/responses are sent on an existing TCP connection is a courtesy, but not a requirement in HTTP/1.1. Both client and server are free to close the connection instead. In fact, such closing is needed when a client and/or server opportunistically keeps a HTTP/1.1 connection open only to later realize that nothing more needs to be requested (in the client case) or that the connection needs to be closed, perhaps to avoid running out of file descriptors or ports (in the server case). For details, read Section 8 of RFC 2616, particularly 8.1.4 Practical Considerations.
According to write(2):
EPIPE fd is connected to a pipe or socket whose reading end is closed. When this happens
the writing process will also receive a SIGPIPE signal. (Thus, the write return
value is seen only if the program catches, blocks or ignores this signal.)
This means the client has closed its file descriptor and would be unable to read() the data sent through this connection. Note that due to buffering and the internals of the TCP protocol, there may be a delay before EPIPE is returned. When it is, you should stop trying to send data on that connection and close the fd.
Before starting any tests, your test harness checks if your server is running by connecting, then disconnecting from it. Your server must not crash when that happens (the book's tiny.c does). You'd detect that if, after accepting a client, the very first read()/recv() call returns 0, indicating EOF on that socket. Make sure you handle this correctly.
Handling both IPv6 and IPv4 clients is (surprisingly) complicated, even over a decade after the IPv6 programming model was developed. Nevertheless, it's not acceptable for network programs today to be IPv4 only.
The first consideration is that your code must work on a machine that has only a IPv4 address, on machine that has only a global IPv6 address, and on a machine that has both. (We don't ask that you support link-local IPv6 addresses, which would actually require additional programming effort). Thus, your program needs to learn which addresses are available on a given system. It uses getaddrinfo() for that, with AI_PASSIVE set, as shown in Drepper's Userlevel IPv6 Programming Introduction.
getaddrinfo() will return a list of addresses (typically one or two). You need to use the addresses returned in a certain manner, described below. First, note that there's (apparently) no guarantee as to the order in which the addresses are returned, as I note in this email, which contradicts the statement Drepper makes on his page. I believe Drepper may have wrongly interpreted RFC 3484, or couldn't convince the maintainers of libc for the major Linux distributions to accept his interpretation, but in any event, on our current CentOS 6 machines which have global IPv4 and IPv6 connectivity, the IPv4 address is returned before the IPv6 address, contrary to Drepper's statement.
There are two approaches of dealing with IPv4 and IPv6. The first approach is to keep them separate. Two separate sockets, separately bound via bind(), with separate calls to listen() and separate calls to accept(). Note that this approach requires multiple threads (or a form of I/O multiplexing such as select()/poll()), because you must avoid the following pitfall:
while (1) { // server loop
int ipv4client = accept(ipv4socket, ....);
handleConnection (ipv4client);
int ipv6client = accept(ipv6socket, ....);
handleConnection (ipv6client);
}
In the above loop, while we wait for an IPv4 client, we cannot accept and serve pending IPv6 clients, and vice versa. Instead, you need one thread for IPv4 accept and one for IPv6 accept. In this project, since you're using multiple threads anyway, implementing this may not be such a burden.
The second approach is to use the so-called dual-bind feature. This feature was introduced to make porting existing servers from IPv4 to IPv6 simpler by allowing the application to just use 1 socket to accept both IPv4 and IPv6 clients. This socket must be bound to the IPv6 address for this to work. If a socket is dual-bound to IPv6/IPv4, a subsequent attempt to bind it to IPv4 will fail with EADDRINUSE. (And vice versa: if the IPv4 port is already bound to another socket, an attempt to dual-bind() to the IPv6 will fail with EADDRINUSE!)
On most Linux systems, including ours, dual-bind is enabled by default for a socket. That is, if you bind the socket to IPv6, it'll automatically be bound to IPv6 and IPv4. There's an option to turn dual-bind off for a given socket. You must turn it off if you wish to use the two socket approach before calling bind(), like so:
// 'pinfo' obtained from getaddrinfo()
// do not dual-bind if this is an IPv6 address
if (pinfo->ai_family == AF_INET6) {
int on = 1;
if (setsockopt(s, IPPROTO_IPV6, IPV6_V6ONLY, (void *)&on, sizeof(on)) == -1)
perror("setsockopt");
}
// now bind(s, pinfo->ai_addr, ...)
So, in summary:
Approach (1): exploit dual-bind. Bind a single-socket, use a single thread to do the accept. Caveat: you can't assume the IPv6 address is before the IPv4 address in the list. Drepper's technique does not work on CentOS 6. Keep in mind your code should still work on any configuration (IPv4-only, IPv6-only, dual-stack). You may need to iterate through the returned list from getaddrinfo twice, first to detect if there's an IPv6 in there, and if so, bind to it first (and only to it), else bind to the IPv4.
Approach (2): handle protocols separately, with two sockets. You need 2 threads (could be tasks in your thread pool, but don't forget that they are long-running, so you'll want to increase the number of threads in the pool since 2 threads will be running the accept() loops). And you need to avoid accidental dual-bind by explicitly turning on IPV6_V6ONLY for the socket you intend to bind to IPv6, and for that socket only.