Notes on HTTP and DNS from Various Sources
Today's objective is to review basic protocols necessary to understand for pentesters. These include Hyper Text Transfer Protocol (HTTP) and Domain Name System (DNS).
To accomplish this objective, HTTP uses a request-response method. This method works with the client-server model in that a user agent (i.e. web browser) submits an HTTP request to a web server. The server then returns the requested resources in the form of HTML files and any other component necessary for facilitating the user's request. Additionally, a response message is returned to the client that provides that status related to the request. HTTP uses status codes to provide more context about the request. These status codes are as follows:
1×× Informational
100 Continue
101 Switching Protocols
102 Processing
2×× Success
200 OK
201 Created
202 Accepted
203 Non-authoritative Information
204 No Content
205 Reset Content
206 Partial Content
207 Multi-Status
208 Already Reported
226 IM Used
3×× Redirection
300 Multiple Choices
301 Moved Permanently
302 Found
303 See Other
304 Not Modified
305 Use Proxy
307 Temporary Redirect
308 Permanent Redirect
4×× Client Error
400 Bad Request
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 Not Acceptable
407 Proxy Authentication Required
408 Request Timeout
409 Conflict
410 Gone
411 Length Required
412 Precondition Failed
413 Payload Too Large
414 Request-URI Too Long
415 Unsupported Media Type
416 Requested Range Not Satisfiable
417 Expectation Failed
418 I'm a teapot
421 Misdirected Request
422 Unprocessable Entity
423 Locked
424 Failed Dependency
426 Upgrade Required
428 Precondition Required
429 Too Many Requests
431 Request Header Fields Too Large
444 Connection Closed Without Response
451 Unavailable For Legal Reasons
499 Client Closed Request
5×× Server Error
500 Internal Server Error
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
506 Variant Also Negotiates
507 Insufficient Storage
508 Loop Detected
510 Not Extended
511 Network Authentication Required
599 Network Connect Timeout Error
Downside: The BA mechanism provides no confidentiality protection for the transmitted credentials. They are merely encoded with Base64 in transit, but not encrypted or hashed in any way. Because the BA field has to be sent in the header of each HTTP request, the web browser needs to cache credentials for a reasonable period of time to avoid constantly prompting the user for their username and password.
Digest Access Authentication: one of the agreed-upon methods a web server can use to negotiate credentials, such as username or password, with a user's web browser. This can be used to confirm the identity of a user before sending sensitive information, such as online banking transaction history. It applies a hash function to the username and password before sending them over the network. In contrast, basic access authentication uses the easily reversible Base64 encoding instead of encryption, making it non-secure unless used in conjunction with TLS. The password is not sent clear to the server, thus preventing Phishing attacks if the user is tricked into accessing the wrong web site.
Disadvantages: Digest access authentication is intended to replace unencrypted HTTP basic access authentication. It is not, however, intended to replace strong authentication protocols, such as public-key or Kerberos authentication. Also, Digest access authentication is vulnerable to a man-in-the-middle (MITM) attack.For example, a MITM attacker could tell clients to use basic access authentication or legacy RFC2069 digest access authentication mode. To extend this further, digest access authentication provides no mechanism for clients to verify the server's identity. Some servers require passwords to be stored using reversible encryption. It prevents the use of a strong password hash (such as bcrypt) when storing passwords (since either the password, or the digested username, realm and password must be recoverable). Also, since the MD5 algorithm is not allowed in FIPS, HTTP Digest authentication will not work with FIPS-certified crypto modules.
GET
The GET method requests a representation of the specified resource. Requests using GET should only retrieve data.
HEAD
The HEAD method asks for a response identical to that of a GET request, but without the response body.
POST
The POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server
PUT
The PUT method replaces all current representations of the target resource with the request payload.
DELETE
The DELETE method deletes the specified resource.
CONNECT
The CONNECT method establishes a tunnel to the server identified by the target resource.
OPTIONS
The OPTIONS method is used to describe the communication options for the target resource.
TRACE
The TRACE method performs a message loop-back test along the path to the target resource.
PATCH
The PATCH method is used to apply partial modifications to a resource.
Some of the methods (for example, HEAD, GET, OPTIONS and TRACE) are, by convention, defined as safe, which means they are intended only for information retrieval and should not change the state of the server. In other words, they should not have side effects, beyond relatively harmless effects such as logging, caching, the serving of banner advertisements or incrementing a web counter. By contrast, methods such as POST, PUT, DELETE and PATCH are intended for actions that may cause side effects either on the server, or external side effects such as financial transactions or transmission of email. Such methods are therefore not usually used by conforming web robots or web crawlers; some that do not conform tend to make requests without regard to context or consequences.
The most common types of records stored in the DNS database are for Start of Authority (SOA), IP addresses (A and AAAA), SMTP mail exchangers (MX), name servers (NS), pointers for reverse DNS lookups (PTR), and domain name aliases (CNAME). Although not intended to be a general purpose database, DNS can store records for other types of data for either automatic lookups, such as DNSSEC records, or for human queries such as responsible person (RP) records. As a general purpose database, the DNS has also been used in combating unsolicited email (spam) by storing a real-time blackhole list. The DNS database is traditionally stored in a structured zone file.
Additionally, it delegates the responsibility of assigning domain names and mapping those names to Internet resources by designating authoritative name servers for each domain. Network administrators may delegate authority over sub-domains of their allocated name space to other name servers. This mechanism provides distributed and fault tolerant service and was designed to avoid a single large central database.
The Domain Name System also specifies the technical functionality of the database service that is at its core. It defines the DNS protocol, a detailed specification of the data structures and data communication exchanges used in the DNS, as part of the Internet Protocol Suite. Historically, other directory services preceding DNS were not scalable to large or global directories as they were originally based on text files, prominently the HOSTS.TXT resolver.
It is important to note that these definitions were taken directly from Wikipedia and only serve as a point of reference. They are not intended to be passed off as anything other than a repository to revisit and as a single resource point.
https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
https://en.wikipedia.org/wiki/Digest_access_authentication
https://en.wikipedia.org/wiki/Basic_access_authentication
Hyper Text Transfer Protocol (HTTP): Ports 80 and 8080
HTTP is an application layer (Layer 1) protocol. It uses hyperlinking in between nodes containing text. That is, is provides the protocol necessary for exchanging or transferring hypertext.To accomplish this objective, HTTP uses a request-response method. This method works with the client-server model in that a user agent (i.e. web browser) submits an HTTP request to a web server. The server then returns the requested resources in the form of HTML files and any other component necessary for facilitating the user's request. Additionally, a response message is returned to the client that provides that status related to the request. HTTP uses status codes to provide more context about the request. These status codes are as follows:
1×× Informational
100 Continue
101 Switching Protocols
102 Processing
2×× Success
200 OK
201 Created
202 Accepted
203 Non-authoritative Information
204 No Content
205 Reset Content
206 Partial Content
207 Multi-Status
208 Already Reported
226 IM Used
3×× Redirection
300 Multiple Choices
301 Moved Permanently
302 Found
303 See Other
304 Not Modified
305 Use Proxy
307 Temporary Redirect
308 Permanent Redirect
4×× Client Error
400 Bad Request
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 Not Acceptable
407 Proxy Authentication Required
408 Request Timeout
409 Conflict
410 Gone
411 Length Required
412 Precondition Failed
413 Payload Too Large
414 Request-URI Too Long
415 Unsupported Media Type
416 Requested Range Not Satisfiable
417 Expectation Failed
418 I'm a teapot
421 Misdirected Request
422 Unprocessable Entity
423 Locked
424 Failed Dependency
426 Upgrade Required
428 Precondition Required
429 Too Many Requests
431 Request Header Fields Too Large
444 Connection Closed Without Response
451 Unavailable For Legal Reasons
499 Client Closed Request
5×× Server Error
500 Internal Server Error
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
505 HTTP Version Not Supported
506 Variant Also Negotiates
507 Insufficient Storage
508 Loop Detected
510 Not Extended
511 Network Authentication Required
599 Network Connect Timeout Error
HTTP Session
An HTTP client initiates a request by establishing a Transmission Control Protocol (TCP) connection to pot 80 or 8080 on a server. Once the server receives the request, it sends back a message and the status (see codes above). The message itself is generally the requested resource(s), although an error message may also be returned if applicable.HTTP Authentication Schemes
Basic Authentication: a method for an HTTP user agent to provide a user name and password when making a request. It doesn't require cookies, session identifiers, or login pages; rather, it uses standard fields in the HTTP header, removing the need for handshakes.Downside: The BA mechanism provides no confidentiality protection for the transmitted credentials. They are merely encoded with Base64 in transit, but not encrypted or hashed in any way. Because the BA field has to be sent in the header of each HTTP request, the web browser needs to cache credentials for a reasonable period of time to avoid constantly prompting the user for their username and password.
Digest Access Authentication: one of the agreed-upon methods a web server can use to negotiate credentials, such as username or password, with a user's web browser. This can be used to confirm the identity of a user before sending sensitive information, such as online banking transaction history. It applies a hash function to the username and password before sending them over the network. In contrast, basic access authentication uses the easily reversible Base64 encoding instead of encryption, making it non-secure unless used in conjunction with TLS. The password is not sent clear to the server, thus preventing Phishing attacks if the user is tricked into accessing the wrong web site.
Disadvantages: Digest access authentication is intended to replace unencrypted HTTP basic access authentication. It is not, however, intended to replace strong authentication protocols, such as public-key or Kerberos authentication. Also, Digest access authentication is vulnerable to a man-in-the-middle (MITM) attack.For example, a MITM attacker could tell clients to use basic access authentication or legacy RFC2069 digest access authentication mode. To extend this further, digest access authentication provides no mechanism for clients to verify the server's identity. Some servers require passwords to be stored using reversible encryption. It prevents the use of a strong password hash (such as bcrypt) when storing passwords (since either the password, or the digested username, realm and password must be recoverable). Also, since the MD5 algorithm is not allowed in FIPS, HTTP Digest authentication will not work with FIPS-certified crypto modules.
HTTP Request Methods
HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. Although they can also be nouns, these request methods are sometimes referred to as HTTP verbs. Each of them implements a different semantic, but some common features are shared by a group of them: e.g. a request method can be safe, idempotent, or cacheable. The following are all of the request methods supported by the HTTP/1.1 specification:GET
The GET method requests a representation of the specified resource. Requests using GET should only retrieve data.
HEAD
The HEAD method asks for a response identical to that of a GET request, but without the response body.
POST
The POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server
PUT
The PUT method replaces all current representations of the target resource with the request payload.
DELETE
The DELETE method deletes the specified resource.
CONNECT
The CONNECT method establishes a tunnel to the server identified by the target resource.
OPTIONS
The OPTIONS method is used to describe the communication options for the target resource.
TRACE
The TRACE method performs a message loop-back test along the path to the target resource.
PATCH
The PATCH method is used to apply partial modifications to a resource.
Some of the methods (for example, HEAD, GET, OPTIONS and TRACE) are, by convention, defined as safe, which means they are intended only for information retrieval and should not change the state of the server. In other words, they should not have side effects, beyond relatively harmless effects such as logging, caching, the serving of banner advertisements or incrementing a web counter. By contrast, methods such as POST, PUT, DELETE and PATCH are intended for actions that may cause side effects either on the server, or external side effects such as financial transactions or transmission of email. Such methods are therefore not usually used by conforming web robots or web crawlers; some that do not conform tend to make requests without regard to context or consequences.
Domain Name System (DNS): Port 53
The following is the most relevant information about DNS taken from Wikipedia:
The Domain Name System (DNS) translates more readily memorized domain names to the numerical IP addresses needed for locating and identifying computer services and devices with the underlying network protocols. The Internet maintains two principal namespaces, the domain name hierarchy and the Internet Protocol (IP) address spaces. DNS maintains the domain name hierarchy and provides translation services between it and the address spaces. Internet name servers and a communication protocol implement the Domain Name System. A DNS name server is a server that stores the DNS records for a domain; a DNS name server responds with answers to queries against its database.The most common types of records stored in the DNS database are for Start of Authority (SOA), IP addresses (A and AAAA), SMTP mail exchangers (MX), name servers (NS), pointers for reverse DNS lookups (PTR), and domain name aliases (CNAME). Although not intended to be a general purpose database, DNS can store records for other types of data for either automatic lookups, such as DNSSEC records, or for human queries such as responsible person (RP) records. As a general purpose database, the DNS has also been used in combating unsolicited email (spam) by storing a real-time blackhole list. The DNS database is traditionally stored in a structured zone file.
Additionally, it delegates the responsibility of assigning domain names and mapping those names to Internet resources by designating authoritative name servers for each domain. Network administrators may delegate authority over sub-domains of their allocated name space to other name servers. This mechanism provides distributed and fault tolerant service and was designed to avoid a single large central database.
The Domain Name System also specifies the technical functionality of the database service that is at its core. It defines the DNS protocol, a detailed specification of the data structures and data communication exchanges used in the DNS, as part of the Internet Protocol Suite. Historically, other directory services preceding DNS were not scalable to large or global directories as they were originally based on text files, prominently the HOSTS.TXT resolver.
It is important to note that these definitions were taken directly from Wikipedia and only serve as a point of reference. They are not intended to be passed off as anything other than a repository to revisit and as a single resource point.
Sources
https://httpstatuses.comhttps://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
https://en.wikipedia.org/wiki/Digest_access_authentication
https://en.wikipedia.org/wiki/Basic_access_authentication
Comments
Post a Comment