Commit graph

351 commits

Author SHA1 Message Date
Benjamin Gruenbaum
a8afe26fca stream: add drop and take
This adds the `drop` and `take` methods to readable streams allowing
users easily drop and take items from the stream.

This continues the iterator-helper proposal alignment task.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/41630
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-23 18:43:16 +02:00
Benjamin Gruenbaum
ca489497be
stream: never flatten on toArray
PR-URL: https://github.com/nodejs/node/pull/41615
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-01-23 08:19:53 +00:00
Benjamin Gruenbaum
cc8931a916 stream: support flatMap
Support the `flatMap` method from the iterator helper TC39 proposal on
readable streams.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/41612
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-23 10:13:17 +02:00
Rich Trott
e6fdcbe534 stream: check for null instead of falsy in loops
Check for null in while loops. This is preparing the code for
the no-cond-assign ESLint rule.

PR-URL: https://github.com/nodejs/node/pull/41614
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2022-01-22 19:38:06 -08:00
Benjamin Gruenbaum
5badf46f2a stream: support some and every
This continues on the iterator-helpers work by adding `.some` and
`.every` to readable streams.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/41573
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-21 12:10:17 +02:00
Benjamin Gruenbaum
5a407d606a stream: add toArray
Add the toArray method from the TC39 iterator helper proposal to
Readable streams. This also enables a common-use case of converting a
stream to an array.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/41553
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-20 11:24:29 +02:00
Benjamin Gruenbaum
3f0bcfb203 stream: add forEach method
Add a `forEach` method to readable streams to enable concurrent
iteration and align with the iterator-helpers proposal.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/41445
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
2022-01-17 14:25:04 +02:00
Rich Trott
20347d5154
stream: avoid function call where possible
Instead of assigning a boolean, move the function call that assigns a
value to the boolean to the only place that boolean is checked. This
avoids the function call in cases where it is not needed.

Refs: https://github.com/nodejs/node/pull/41488#pullrequestreview-850626528

PR-URL: https://github.com/nodejs/node/pull/41534
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-17 05:31:30 +00:00
Rich Trott
f7be6ab042
stream: remove always-false condition check
Remove comparison to null of variable guaranteed to be a boolean.

PR-URL: https://github.com/nodejs/node/pull/41488
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
2022-01-15 00:07:46 +00:00
Rich Trott
8c3637cd35
stream: fix error-path function call
The `onFinish()` function takes a single argument. The two extra
arguments passed here are already in the function scope, and may result
in the error being mishandled.

PR-URL: https://github.com/nodejs/node/pull/41433
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2022-01-11 02:53:21 +00:00
Robert Nagy
48e784043d stream: don't emit finish after destroy
PR-URL: https://github.com/nodejs/node/pull/40852
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2022-01-03 16:32:44 +01:00
Benjamin Gruenbaum
55c5120b07 stream: add filter method to readable
This continues the work in https://github.com/nodejs/node/pull/40815 to
make streams compatible with upcoming ECMAScript language features. It
adds an experimental `filter` api to streams and tests/docs for it.

See https://github.com/tc39/proposal-iterator-helpers/

Co-Authored-By: Robert Nagy <ronagy@icloud.com>

PR-URL: https://github.com/nodejs/node/pull/41354
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-01-02 09:52:50 +02:00
Robert Nagy
a698c49993 stream: add isReadable helper
PR-URL: https://github.com/nodejs/node/pull/41199
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-12-29 20:42:10 +01:00
Benjamin Gruenbaum
b97b81d4ec stream: add map method to Readable
Implement the map method on readable stream. This starts the alignment
with the tc39-iterator-helpers proposal and adds a `.map` method to
every Node.js readable stream.

Co-Authored-By: Robert Nagy <ronag@icloud.com>

PR-URL: https://github.com/nodejs/node/pull/40815
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-12-29 20:32:36 +01:00
Robert Nagy
752d75d8bc stream: add isErrored helper
Refs: https://github.com/nodejs/undici/pull/1134

PR-URL: https://github.com/nodejs/node/pull/41121
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2021-12-16 14:28:31 +01:00
Robert Nagy
1fa507f098 stream: allow readable to end early without error
PR-URL: https://github.com/nodejs/node/pull/40881
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2021-12-06 20:42:12 +01:00
James M Snell
af28b0d540
stream: use cause options in AbortError constructors
Signed-off-by: James M Snell <jasnell@gmail.com>

PR-URL: https://github.com/nodejs/node/pull/41008
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-12-02 12:14:10 -08:00
Robert Nagy
40a52bde95
stream: cleanup eos
PR-URL: https://github.com/nodejs/node/pull/40998
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-12-01 01:50:24 +00:00
Robert Nagy
37f1dd9ccd stream: drain Transform with 0 highWaterMark
Fixes: https://github.com/nodejs/node/issues/40935

PR-URL: https://github.com/nodejs/node/pull/40947
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
2021-11-27 18:09:41 +01:00
Robert Nagy
49b8c4f08b
stream: stricter isReadableNodeStream
Fixes: https://github.com/nodejs/node/issues/40938

PR-URL: https://github.com/nodejs/node/pull/40941
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-11-26 17:33:05 +00:00
Robert Nagy
8ee4e672ec stream: pipeline with end option
Currently pipeline cannot fully replace pipe due
to the missing end option. This PR adds the end
option to the promisified pipeline method.

PR-URL: https://github.com/nodejs/node/pull/40886
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-11-22 10:52:23 +01:00
Matteo Collina
cb75dec299
stream: fix finished regression when working with legacy Stream
Signed-off-by: Matteo Collina <hello@matteocollina.com>

PR-URL: https://github.com/nodejs/node/pull/40858
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-11-22 09:32:02 +00:00
Robert Nagy
340b770d3f stream: unify writableErrored and readableErrored
Both of these should always refer to the same error, hence
there is no reason to separate them.

PR-URL: https://github.com/nodejs/node/pull/40799
Refs: https://github.com/nodejs/node/pull/40696
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-11-18 14:31:43 +01:00
Robert Nagy
0a3a60130c stream: pipeline should drain empty readable
This simplifies some cases where the last stream is a Duplex
without any expected output.

await pipeline(readable, duplex)

PR-URL: https://github.com/nodejs/node/pull/40654
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2021-11-18 11:02:00 +01:00
Robert Nagy
415726b8c4 stream: add writableAborted
PR-URL: https://github.com/nodejs/node/pull/40802
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2021-11-18 07:40:13 +01:00
Robert Nagy
640bfb8d70 stream: fix non readable Duplex readableAborted
PR-URL: https://github.com/nodejs/node/pull/40801
Fixes: https://github.com/nodejs/node/issues/40800
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-11-15 14:45:46 +02:00
Robert Nagy
afe460ec9e stream: allow calling callback before promise
Refs: https://github.com/nodejs/node/issues/39535

PR-URL: https://github.com/nodejs/node/pull/40772
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-11-14 15:52:22 +02:00
Robert Nagy
f2170253b6 stream: add errored and closed props
PR-URL: https://github.com/nodejs/node/pull/40696
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-11-13 11:19:32 +02:00
Robert Nagy
47fb8676ff stream: remove no longer necessary ComposeDuplex
PR-URL: https://github.com/nodejs/node/pull/40545
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-10-23 22:23:35 +00:00
Mestery
b920a10162
stream: support array of streams in promises pipeline
Fixes: https://github.com/nodejs/node/issues/40191

PR-URL: https://github.com/nodejs/node/pull/40193
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-10-20 19:47:34 +08:00
Robert Nagy
0f78d2600b stream: fix fromAsyncGen
Fixes: https://github.com/nodejs/node/issues/40497

PR-URL: https://github.com/nodejs/node/pull/40499
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-10-19 13:55:06 +00:00
Maledong
8900a4663f
lib: remove useless statement
There's no need to set 'false' to 'isAsync' because we've declared it
before.

PR-URL: https://github.com/nodejs/node/pull/39983
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
2021-09-18 16:05:59 +02:00
Rongjian Zhang
0c6f345cda stream: fix highwatermark threshold and add the missing error
1. Fix highwatermark threshold: `< 1GiB` -> `<= 1GiB`
2. Add the missing error: Size out of range

Update test/parallel/test-streams-highwatermark.js

Co-authored-by: Darshan Sen <raisinten@gmail.com>

PR-URL: https://github.com/nodejs/node/pull/38700
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2021-09-07 14:28:20 +00:00
Robert Nagy
c04d621ecc stream: add signal support to pipeline generators
Generators in pipeline must be able to be aborted or pipeline
can deadlock.

PR-URL: https://github.com/nodejs/node/pull/39067
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-08-25 17:44:28 +02:00
Robert Nagy
4832d1c02c stream: add isDisturbed helper
Adds a helper util used to determine whether a stream has been
disturbed (read or cancelled).

Refs: https://github.com/nodejs/node/issues/39627

PR-URL: https://github.com/nodejs/node/pull/39628
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-08-06 07:45:38 +02:00
Robert Nagy
0e841b45c2 stream: don't emit 'data' after 'error' or 'close'
As per doc we should not emit further events after 'close' and
only 'close' after 'error'.

Fixes: https://github.com/nodejs/node/issues/39630

PR-URL: https://github.com/nodejs/node/pull/39639
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2021-08-06 07:44:09 +02:00
Mestery
712059a232 stream: clean endWritableNT
The `state` argument was unused.

PR-URL: https://github.com/nodejs/node/pull/39645
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com>
2021-08-05 16:01:55 +00:00
Szymon Marczak
ef992f6de9 stream: do not emit end on readable error
PR-URL: https://github.com/nodejs/node/pull/39607
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-08-05 09:58:01 +00:00
Robert Nagy
533cafcf7e stream: duplexify
PR-URL: https://github.com/nodejs/node/pull/39519
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-08-03 10:47:10 +02:00
Robert Nagy
ac621ff5e7 stream: add readableDidRead if has been read from
Adds did read accessor used to determine whether a readable has been
read from.

PR-URL: https://github.com/nodejs/node/pull/39589
Refs: https://github.com/nodejs/undici/pull/907
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-08-02 05:59:18 -07:00
Robert Nagy
79f4d5a345 stream: fix toWeb typo
PR-URL: https://github.com/nodejs/node/pull/39496
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
2021-07-25 19:32:51 +02:00
Robert Nagy
e579acb8dc stream: add stream.compose
Refs: #32020

PR-URL: https://github.com/nodejs/node/pull/39029
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2021-07-19 08:14:32 +02:00
Robert Nagy
efd40eadab stream: forward errored to callback
Refs: https://github.com/nodejs/node/issues/39356

PR-URL: https://github.com/nodejs/node/pull/39364
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-07-14 13:05:10 +02:00
Robert Nagy
8306051001 stream: add readableDidRead
Adds readableDidRead to streams and applies usage to http, http2 and quic.

PR-URL: https://github.com/nodejs/node/pull/36820
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2021-07-14 12:01:41 +02:00
James M Snell
a99c230305
stream: implement streams to webstreams adapters
Experimental adapters for the webstreams API

Signed-off-by: James M Snell <jasnell@gmail.com>

PR-URL: https://github.com/nodejs/node/pull/39134
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-07-13 10:03:16 -07:00
Robert Nagy
09d8c0c8d2 stream: destroy readable on read error
PR-URL: https://github.com/nodejs/node/pull/39342
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-07-12 12:08:43 +02:00
Robert Nagy
a5dec3a470 stream: validate abort signal
PR-URL: https://github.com/nodejs/node/pull/39346
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-07-12 12:06:57 +02:00
Robert Nagy
f2f6872b5c stream: cleanup async handling
Cleanup async stream method handling.

PR-URL: https://github.com/nodejs/node/pull/39329
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-07-11 19:20:01 +02:00
Robert Nagy
7a7ba82234 stream: don't emit prefinish after error or close
PR-URL: https://github.com/nodejs/node/pull/39332
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-07-11 18:41:17 +02:00
Robert Nagy
5960f16713 tls: move legacy code into own file
PR-URL: https://github.com/nodejs/node/pull/39333
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2021-07-11 18:40:18 +02:00