Understanding HTTP: The Backbone of Web Communication
HTTP (Hypertext Transfer Protocol) is the foundation of data communication on the World Wide Web. As a crucial component in Over-the-Top (OTT) and Real-Time Communication (RTC) technologies, understanding HTTP is essential for developers and engineers working in these fields. This blog post will delve into the intricacies of HTTP, its structure, and its application in streaming technologies like HLS (HTTP Live Streaming).
What is HTTP?
HTTP is an application-layer protocol built on top of TCP. It's the most widely used protocol on the internet, facilitating the transfer of hypertext documents, images, videos, and other multimedia content. HTTPS (HTTP Secure) is a secure version of HTTP that uses SSL/TLS encryption for data transfer.
HTTP URL Structure
HTTP resources are identified using Uniform Resource Locators (URLs). A typical HTTP URL has the following structure:
http://host[:port]/[path/]filename[?param1=value1¶m2=value2][#fragment]
http:
//
: Indicates the use of the HTTP protocol (orhttps:
//
for HTTPS)host
: The domain name or IP address of the serverport
: The port number (default is 80 for HTTP, 443 for HTTPS)path
: The path to the resource on the serverfilename
: The name of the resourceparam
,value
: Query parametersfragment
: A specific part of the resource, often called an anchor
HTTP Communication
HTTP communication involves requests from clients (User Agents) and responses from servers (Origin Servers). Intermediaries like proxy servers, gateways, or tunnels may exist between the client and server.
HTTP Request Structure
An HTTP request consists of three parts:
- Request Line
- Message Headers
- Request Body
Example of an HTTP request:
POST /index.php HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
Accept: text/html,application/xhtml+xml, application/xml; q=0.9,*/*; q=0.8
Accept-Language: zh-cn,zh;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost/
Content-Length: 26
Content-Type: application/x-www-form-urlencoded
username=abc&password=1234
HTTP Response Structure
An HTTP response also consists of three parts:
- Status Line
- Message Headers
- Response Body
Example of an HTTP response:
HTTP/1.1 200 OK
Date: Sun, 17 Mar 2013 08:12:54 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 4393
Keep-Alive: timeout=5, max=100
Connection: keep-alive
Content-Type: text/html; charset=utf-8
<html>
<head>
<title>HTTP Response Example</title>
</head>
<body>
Hello HTTP!
</body>
</html>
HTTP Status Codes
HTTP status codes are three-digit numbers returned by a server in response to a client's request. They are grouped into five classes:
Status Code | Description | Meaning |
1XX | Informational | Request received, continuing process |
2XX | Successful | The action was successfully received, understood, and accepted |
3XX | Redirection | Further action needs to be taken in order to complete the request |
4XX | Client Error | The request contains bad syntax or cannot be fulfilled |
5XX | Server Error | The server failed to fulfill an apparently valid request |
Here are some common status codes:
Status Code | Text | Description |
200 | OK | Request successful |
206 | Partial Content | Partial content delivered |
301 | Moved Permanently | Requested URL has been permanently moved |
302 | Found | Temporary URL redirection |
304 | Not Modified | Resource not modified since last request |
400 | Bad Request | Server cannot process due to client error |
403 | Forbidden | Server refuses to authorize the request |
404 | Not Found | Requested resource could not be found |
500 | Internal Server Error | Generic error message when server fails |
502 | Bad Gateway | Server received an invalid response from upstream server |
503 | Service Unavailable | Server is temporarily unable to handle the request |
For more information, you can refer to
HTTP in Streaming: HLS (HTTP Live Streaming)
HLS is a streaming protocol developed by Apple that uses HTTP for media delivery. It works by breaking the overall stream into a sequence of small HTTP-based file downloads.
HLS Workflow
- The client first retrieves a master playlist file (M3U8).
- The M3U8 file contains URLs of media segment files (TS files).
- The client then downloads these TS files via HTTP.
- For live streaming, the client continuously updates the M3U8 file to get new TS file URLs.
M3U8 File Structure
M3U8 files are text-based and consist of a series of tags. Here's an example:
#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10,
http://media.example.com/segment0.ts
#EXTINF:10,
http://media.example.com/segment1.ts
#EXTINF:10,
http://media.example.com/segment2.ts
#EXT-X-ENDLIST
Here's a table of common M3U8 tags:
Tag | Meaning |
#EXTM3U | Must be the first line of every M3U8 file |
#EXTINF | Specifies the duration of the TS file |
#EXT-X-TARGETDURATION | Specifies the maximum duration of any TS file |
#EXT-X-MEDIA-SEQUENCE | Specifies the sequence number for the first TS file |
#EXT-X-KEY | Specifies the encryption method for TS files |
#EXT-X-PROGRAM-DATE-TIME | Specifies the date/time of the first sample in a TS file |
#EXT-X-ALLOW-CACHE | Specifies whether the client may cache the downloaded media files |
#EXT-X-PLAYLIST-TYPE | Provides information about the playlist type (EVENT or VOD) |
#EXT-X-ENDLIST | Indicates the end of the M3U8 file |
HTTP Dynamic Live Streaming (HDL)
HTTP Dynamic Live Streaming (HDL), more commonly known as HTTP-FLV, is a streaming media protocol developed by Adobe. It leverages HTTP for the online transmission of FLV (Flash Video) files, making it an important technology in the realm of web-based streaming.
HDL's working principle is similar to HLS (HTTP Live Streaming). Both protocols use HTTP to retrieve relevant files and subsequently fetch the corresponding audio and video stream data. However, where HLS uses M3U8 and TS files, HDL utilizes SWF (Shockwave Flash) and FLV files. These file formats, designed by Adobe, traditionally required Flash technology support, which limited their direct use in HTML5 environments. (In browsers supporting MSE, libraries like flv.js can be used to enable support, though compatibility issues should be considered.)
SWF (Shockwave Flash) is a proprietary format developed by Macromedia (later acquired by Adobe) for their Flash animation design software. It has been widely used in web design and animation production. The structure of an SWF file is as follows:
After the Header, there are several tagged data blocks. The Tag format is consistent with FLV encapsulation. During online playback, typically only one SWF file needs to be retrieved.
The HDL playback process follows these steps:
- Retrieve the SWF file via HTTP
- Use the SWF file to obtain FLV files
- Decapsulate, decode, and display the FLV file content
- Continuously fetch and process FLV file data, decoding and displaying it until the EndTag of the SWF file is reached
This process demonstrates how HDL utilizes HTTP not just for initial file retrieval, but for the entire streaming process, making it an integral part of the protocol's functionality.
For more detailed information about SWF files, you can refer to the Adobe Developer Center: http://www.adobe.com/devnet/swf.html
Conclusion
HTTP forms the backbone of web communication and plays a crucial role in modern streaming technologies like HLS. Understanding its structure, status codes, and application in streaming protocols is essential for developers working in OTT and RTC fields. As the web continues to evolve, HTTP remains a fundamental technology, adapting to new requirements and use cases in the ever-changing landscape of digital communication.