refactor validation code using validate function
PR-URL: https://github.com/nodejs/node/pull/59296
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
This patch implements proxy support for HTTP and HTTPS clients and
agents in the `http` and `https` built-ins`. When NODE_USE_ENV_PROXY
is set to 1, the default global agent would parse the
HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy
settings from the environment variables, and proxy the requests
sent through the built-in http/https client accordingly.
To support this, `http.Agent` and `https.Agent` now accept a few new
options:
- `proxyEnv`: when it's an object, the agent would read and parse
the HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy
properties from it, and apply them based on the protocol it uses
to send requests. This option allows custom agents to
reuse built-in proxy support by composing options. Global agents
set this to `process.env` when NODE_USE_ENV_PROXY is 1.
- `defaultPort` and `protocol`: these allow setting of the default port
and protocol of the agents. We also need these when configuring
proxy settings and deciding whether a request should be proxied.
Implementation-wise, this adds a `ProxyConfig` internal class to handle
parsing and application of proxy configurations. The configuration
is parsed during agent construction. When requests are made,
the `createConnection()` methods on the agents would check whether
the request should be proxied. If yes, they either connect to the
proxy server (in the case of HTTP reqeusts) or establish a tunnel
(in the case of HTTPS requests) through either a TCP socket (if the
proxy uses HTTP) or a TLS socket (if the proxy uses HTTPS).
When proxying HTTPS requests through a tunnel, the connection listener
is invoked after the tunnel is established. Tunnel establishment uses
the timeout of the request options, if there is one. Otherwise it uses
the timeout of the agent.
If an error is encountered during tunnel establishment, an
ERR_PROXY_TUNNEL would be emitted on the returned socket. If the proxy
server sends a errored status code, the error would contain an
`statusCode` property. If the error is caused by timeout, the error
would contain a `proxyTunnelTimeout` property.
This implementation honors the built-in socket pool and socket limits.
Pooled sockets are still keyed by request endpoints, they are just
connected to the proxy server instead, and the persistence of the
connection can be maintained as long as the proxy server respects
connection/proxy-connection or persist by default (HTTP/1.1)
PR-URL: https://github.com/nodejs/node/pull/58980
Refs: https://github.com/nodejs/node/issues/57872
Refs: https://github.com/nodejs/node/issues/8381
Refs: https://github.com/nodejs/node/issues/15620
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This makes it possible to disable the various default headers directly
from the constructor. While this is possible for many use cases by
manually calling removeHeader on the request object instead, when
passing a raw header array to the request constructor the headers are
serialized and prepared to send immediately, and removeHeader cannot
subsequently be used.
With this change, it's now possible to 100% control sent request
headers by passing 'setDefaultHeaders: false' and a raw headers array to
http.request.
PR-URL: https://github.com/nodejs/node/pull/56112
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/52625
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
PR-URL: https://github.com/nodejs/node/pull/33633
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/46867
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Throwing error after checking type is repeated. So replace
it with validate function.
PR-URL: https://github.com/nodejs/node/pull/46101
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
When encountering ERR_UNESCAPED_CHARACTERS on large applications it can
be unclear which request has caused this error. Even when setting
NODE_DEBUG=http there is no information about this error since it's
thrown before any debug logs. This patch adds a debug log that contains
the invalid path.
PR-URL: https://github.com/nodejs/node/pull/45420
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Ensuring every request is assigned to a drained socket or nothing.
Because is has no benifit for a request to be attached to a non
drained socket and it prevents the request from being assigned to
a drained one, which might occur soon or already in the free pool
We achieve this by claiming a socket as free only when the socket
is drained.
PR-URL: https://github.com/nodejs/node/pull/43902
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
There are no clear indicators anyone is using the dtrace USDT probes.
ETW support is very intertwined with the dtrace infrastructure. It's not
clear if anyone uses ETW so to keep things simple it too is removed.
Fixes: https://github.com/nodejs/node/issues/43649
PR-URL: https://github.com/nodejs/node/pull/43652
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
PR-URL: https://github.com/nodejs/node/pull/43580
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
RFC 5246 section-7.2.1 requires that the implementation must immediately
stop reading from the stream, as it is no longer TLS-encrypted. The
underlying stream is permitted to still pump events (and errors) to
other users, but those are now unencrypted, so we should not process
them here. But therefore, we do not want to stop the underlying stream,
as there could be another user of it, but we do need to remove ourselves
as a listener.
Per TLS v1.2, we should have also destroy the TLS state entirely here
(including the writing side), but this was revised in TLS v1.3 to permit
the stream to continue to flush output.
There appears to be some inconsistencies in the way nodejs handles
ownership of the underlying stream, with `TLS.close()` on the write side
also calling shutdown on the underlying stream (thus assuming other
users of the underlying stream are not permitted), while receiving EOF
on the read side leaves the underlying channel open. These
inconsistencies are left for a later person to resolve, if the extra
functionality is needed (as described in #35904). The current goal here
is to the fix the occasional CI exceptions depending on the timing of
these kernel messages through the TCP stack.
PR-URL: https://github.com/nodejs/node/pull/36111
Fixes: https://github.com/nodejs/node/pull/35946
Refs: https://github.com/libuv/libuv/pull/3036
Refs: https://github.com/nodejs/node/issues/35904
Co-authored-by: Momtchil Momtchev <momtchil@momtchev.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
There might be the case of some more data coming through after
the parser has returned an error and we have destroyed the socket.
We should also be removing the 'data' event handler.
Fixes: https://github.com/nodejs/node/issues/40242
PR-URL: https://github.com/nodejs/node/pull/40244
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Unify stream helps into utils.
PR-URL: https://github.com/nodejs/node/pull/39294
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Calling finished before or after a stream has errored or closed
should end up with the same behavior.
PR-URL: https://github.com/nodejs/node/pull/39235
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/38505
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/38598
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Exporting a variable that will be mutated later doesn't work.
Refs: https://github.com/nodejs/node/issues/37937
PR-URL: https://github.com/nodejs/node/pull/37966
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
This reverts commit e2f5bb7574.
Reverted as it caused a significant performance regression.
See: https://github.com/nodejs/node/issues/37937
PR-URL: https://github.com/nodejs/node/pull/37963
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Fix an issue where AbortSignals are registered twice
PR-URL: https://github.com/nodejs/node/pull/37730
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Futher aligns OutgoingMessage with stream.Writable. In particular
re-uses the construct/destroy logic from streams.
Due to a lot of subtle assumptions this PR unfortunately touches
a lot of different parts.
PR-URL: https://github.com/nodejs/node/pull/36816
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Since url.parse() is deprecated, it must not be used inside Node.js.
PR-URL: https://github.com/nodejs/node/pull/36853
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Enable the default `autoDestroy: true` option in IncomingMessage.
Refactor `_http_client` and `_http_server` to remove any manual
destroying/closing of IncomingMessage.
Refactor IncomingMessage `destroy` method to use the standard
implementation of the stream module and move the early termination
event emitting inside of it.
PR-URL: https://github.com/nodejs/node/pull/33035
Refs: https://github.com/nodejs/node/issues/30625
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
