Commit graph

371 commits

Author SHA1 Message Date
Dmitry Stogov
90b7bde615 Use more compact representation for packed arrays.
- for packed arrays we store just an array of zvals without keys.
- the elements of packed array are accessible throuf as ht->arPacked[i]
  instead of ht->arData[i]
- in addition to general ZEND_HASH_FOREACH_* macros, we introduced similar
  familied for packed (ZEND_HASH_PACKED_FORECH_*) and real hashes
  (ZEND_HASH_MAP_FOREACH_*)
- introduced an additional family of macros to access elements of array
  (packed or real hashes) ZEND_ARRAY_ELEMET_SIZE, ZEND_ARRAY_ELEMET_EX,
  ZEND_ARRAY_ELEMET, ZEND_ARRAY_NEXT_ELEMENT, ZEND_ARRAY_PREV_ELEMENT
- zend_hash_minmax() prototype was changed to compare only values

Because of smaller data set, this patch may show performance improvement
on some apps and benchmarks that use packed arrays. (~1% on PHP-Parser)

TODO:
    - sapi/phpdbg needs special support for packed arrays (WATCH_ON_BUCKET).
    - zend_hash_sort_ex() may require converting packed arrays to hash.
2021-11-03 15:18:26 +03:00
Nikita Popov
6d505d4445 Add RETURN/RETVAL_COPY_DEREF() macros
These were missing from the set...

I think quite a few of these usages don't actually need the DEREF,
but I've just kept things as is for now.
2021-07-22 09:44:19 +02:00
Nikita Popov
814a932734 Add ZEND_ACC_NOT_SERIALIZABLE flag
This prevents serialization and unserialization of a class and its
children in a way that does not depend on the zend_class_serialize_deny
and zend_class_unserialize_deny handlers that will be going away
in PHP 9 together with the Serializable interface.

In stubs, `@not-serializable` can be used to set this flag.

This patch only uses the new flag for a handful of Zend classes,
converting the remainder is left for later.

Closes GH-7249.
Fixes bug #81111.
2021-07-19 15:59:11 +02:00
Nikita Popov
4411026217 Close generator already in dtor phase
In the added test case, the Closure ends up being freed before
the generator during GC.

This patch closes the generator (and thus releases the held
closure / execute_data) already during dtor_obj, which will avoid
ordering issues in free_obj. dtor_obj is not always called, but
if it isn't, then we also won't run GC and will free_obj in
reverse construction order.

Fixes oss-fuzz #33947.
2021-05-04 16:59:59 +02:00
Josh Soref
462da6e09c Fix spelling and grammar mistakes
This PR corrects misspellings identified by the check-spelling action.

The misspellings have been reported at jsoref@b6ba3e2#commitcomment-48946465

The action reports that the changes in this PR would make it happy: jsoref@602417c

Closes GH-6822.
2021-04-13 12:09:37 +02:00
Máté Kocsis
1954e59758
Add support for generating class entries from stubs
Closes GH-6289

Co-authored-by: Nikita Popov <nikita.ppv@gmail.com>
2021-01-26 11:50:36 +01:00
Nikita Popov
3e01f5afb1 Replace zend_bool uses with bool
We're starting to see a mix between uses of zend_bool and bool.
Replace all usages with the standard bool type everywhere.

Of course, zend_bool is retained as an alias.
2021-01-15 12:33:06 +01:00
Nikita Popov
22793884b6 Remove some INDIRECT handling in VM 2021-01-06 12:46:31 +01:00
Nikita Popov
a6bcad4a9e Merge branch 'PHP-8.0'
* PHP-8.0:
  Short-circuit get_gc for currently running generator
2020-11-18 12:46:32 +01:00
Nikita Popov
bbc0dd402a Merge branch 'PHP-7.4' into PHP-8.0
* PHP-7.4:
  Short-circuit get_gc for currently running generator
2020-11-18 12:46:12 +01:00
Nikita Popov
99a8ec6e24 Short-circuit get_gc for currently running generator 2020-11-18 12:45:29 +01:00
Nikita Popov
a58e371cef Remove unused single.leaf member 2020-10-22 10:42:26 +02:00
Nikita Popov
dd4a080133 Simplify and fix generator tree management
This makes a number of related changes to the generator tree
management, that should hopefully make it easier to understand,
more robust and faster for the common linear-chain case. Fixes
https://bugs.php.net/bug.php?id=80240, which was the original
motivation here.

 * Generators now only add a ref to their direct parent.
 * Nodes only store their children, not their leafs, which avoids
   any need for leaf updating. This means it's no longer possible
   to fetch the child for a certain leaf, which is something we
   only needed in one place (update_current). If multi-children
   nodes are involved, this will require doing a walk in the other
   direction (from leaf to root). It does not affect the common
   case of single-child nodes.
 * The root/leaf pointers are now seen as a pair. One leaf generator
   can point to the current root. If a different leaf generator is
   used, we'll move the root pointer over to that one. Again, this
   is a cache to make the common linear chain case fast, trees may
   need to scan up the parent link.

Closes GH-6344.
2020-10-22 10:25:25 +02:00
Nikita Popov
c5401854fc Run tidy
This should fix most of the remaining issues with tabs and spaces
being mixed in tests.
2020-09-18 14:28:32 +02:00
Dmitry Stogov
d5d31ea3b3 Cleanup observer API and add JIT support 2020-09-18 12:55:58 +03:00
Bob Weinand
452f7b0d41 Fix use-after-free with yield from in yield_from_multi_tree_single_nodes.phpt
Prevent release of generator children during destruction
2020-09-14 23:07:01 +02:00
Bob Weinand
ad61e141dd Fix crashes with unproper cleaning of repeated yield from
Closes GH-6130
2020-09-14 20:49:24 +02:00
Nikita Popov
174dadf6b4 Don't allow dynamic properties on generators
Noticed this because we leak those properties in GC. This was
never intended to be allowed.
2020-09-07 19:08:04 +02:00
Nikita Popov
ddc2a2d381 Merge branch 'PHP-7.4'
* PHP-7.4:
  Fix throwing of yield from related exceptions into generator
2020-09-02 10:53:44 +02:00
Nikita Popov
c6fd37cee3 Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Fix throwing of yield from related exceptions into generator
2020-09-02 10:53:13 +02:00
Nikita Popov
2e9e706a82 Fix throwing of yield from related exceptions into generator
Use the general zend_generator_throw_exception() helper for this.
Otherwise we don't handle the off-by-one opline correctly (should
we maybe just stop doing that?)

This is a followup to ad750c3bb6,
which fixed a different yield from exception handling problem that
happened to show up in the same test case from oss-fuzz #25321.
Now both issues should be fixed.
2020-09-02 10:52:55 +02:00
Nikita Popov
c6ea0e90c2 Assert there are children in zend_generator_get_child() 2020-09-01 20:37:43 +02:00
Levi Morrison
66c3e900e2 Add zend_observer API
Closes GH-5857.

Co-authored-by: Nikita Popov <nikita.ppv@gmail.com>
Co-authored-by: Sammy Powers <sammyk@datadoghq.com>
2020-09-01 09:59:59 -06:00
Nikita Popov
ad97739eac Merge branch 'PHP-7.4'
* PHP-7.4:
  Fix handling of exception if valid() during yield from
2020-08-31 10:52:07 +02:00
Nikita Popov
b396fb348b Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Fix handling of exception if valid() during yield from
2020-08-31 10:51:49 +02:00
Nikita Popov
ad750c3bb6 Fix handling of exception if valid() during yield from
Fixes oss-fuzz #25296.
2020-08-31 10:51:31 +02:00
George Peter Banyard
fa8d9b1183 Improve type declarations for Zend APIs
Voidification of Zend API which always succeeded
Use bool argument types instead of int for boolean arguments
Use bool return type for functions which return true/false (1/0)
Use zend_result return type for functions which return SUCCESS/FAILURE as they don't follow normal boolean semantics

Closes GH-6002
2020-08-28 15:41:27 +02:00
Nikita Popov
e25aab6426 Fixed bug #79927
We need to unset the AT_FIRST_YIELD flag when yielding from an
array as well.

In the interest of being conservative, I'm applying this only to
PHP 8.
2020-08-11 15:48:40 +02:00
Nikita Popov
d92229d8c7 Implement named parameters
From an engine perspective, named parameters mainly add three
concepts:

 * The SEND_* opcodes now accept a CONST op2, which is the
   argument name. For now, it is looked up by linear scan and
   runtime cached.
 * This may leave UNDEF arguments on the stack. To avoid having
   to deal with them in other places, a CHECK_UNDEF_ARGS opcode
   is used to either replace them with defaults, or error.
 * For variadic functions, EX(extra_named_params) are collected
   and need to be freed based on ZEND_CALL_HAS_EXTRA_NAMED_PARAMS.

RFC: https://wiki.php.net/rfc/named_params

Closes GH-5357.
2020-07-31 15:53:36 +02:00
Max Semenik
2b5de6f839
Remove proto comments from C files
Closes GH-5758
2020-07-06 21:13:34 +02:00
Nikita Popov
271bc689ea Add iterator get_gc function for generators
Closes GH-5787.
2020-07-01 15:17:26 +02:00
Nikita Popov
312201dce4 Add get_gc handle for object iterators
Optional handler with the same semantics as the object handler.
2020-07-01 15:17:22 +02:00
Nikita Popov
89b2483edd Remove generator iterator member
This is probably a leftover from the PHP 5 implementation, where
the iterator was embedded directly in the generator.
2020-06-30 18:26:29 +02:00
Nikita Popov
af4ff75c98 Merge branch 'PHP-7.4'
* PHP-7.4:
  Remove bogus generator iterator dtor
2020-06-30 17:33:51 +02:00
Nikita Popov
d1be10dd7e Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Remove bogus generator iterator dtor
2020-06-30 17:33:33 +02:00
Nikita Popov
187a72d563 Remove bogus generator iterator dtor
Fixes a use-after-free encountered in Symfony's SecurityBundle.
I don't have a reproducer for this, and believe the issue can only
occur if we leak an iterator (the leak is a separate issue).

We should not free the generator iterator here, because we do not
own it. The code that fetched the iterator is responsible for
releasing it. In the rare case where we do hit this code-path,
we cause a use-after-free.
2020-06-30 17:32:42 +02:00
Dmitry Stogov
24a8065f8b Tracing JIT support for include() and generators 2020-06-29 21:40:49 +03:00
Nikita Popov
15846ff115 Add ZVAL_OBJ_COPY macro
For the common ZVAL_OBJ + GC_ADDREF pattern.
This mirrors the existing ZVAL_STR_COPY API.
2020-06-17 16:36:56 +02:00
Nikita Popov
57f408e87c Merge branch 'PHP-7.4'
* PHP-7.4:
  Fixed bug #79657
2020-06-08 11:32:24 +02:00
Nikita Popov
df2db7fcea Fixed bug #79657
Throwing an exception should count as an initialization for this
purpose.
2020-06-08 11:31:28 +02:00
Nikita Popov
00ef092335 Merge branch 'PHP-7.4'
* PHP-7.4:
  Fixed bug #79600
2020-05-15 11:05:00 +02:00
Nikita Popov
d87b0685b5 Fixed bug #79600
Missed unsetting of DO_INIT in one case.
2020-05-15 11:04:40 +02:00
Tyson Andre
d63eca285a
Fix miscellaneous typos in code comments/var names
Closes GH-5501
2020-05-01 15:49:05 +02:00
Nikita Popov
48a34bc120 Add helper APIs for get_gc implementations
get_gc() implementations that need to explore heterogeneous data
currently work by computing how many GC entries they need,
allocating a buffer for that and storing it on the object. This
is inefficient and wastes memory, because the buffer is retained
after the GC run.

This commit adds an API for a single global GC buffer, which can
be reused by get_gc implementations (as only one get_gc call is
ever active at the same time). The GC buffer will automatically
grow during the GC run and be discarded at the end.
2020-04-27 10:48:22 +02:00
Máté Kocsis
33c3691c33
Generate method entries from stubs for Zend classes
Closes GH-5459
2020-04-26 11:13:14 +02:00
Nikita Popov
8ba6a833de Merge branch 'PHP-7.4'
* PHP-7.4:
  Fixed bug #78434
2020-04-09 10:35:50 +02:00
Nikita Popov
823a956855 Fixed bug #78434
The DO_INIT flag, which will skip the first resume on a primed
generator, should always be set when starting to yield from a
new generator, not only when the yield from happens during priming.
2020-04-09 10:33:11 +02:00
Máté Kocsis
8f4f1dea34
Convert zend_parse_parameters_none() to fast ZPP
I've done the conversion in those extensions where fast ZPP is predominant.
2020-01-03 13:23:37 +01:00
Máté Kocsis
e1b57310b1
Use RETURN_THROWS() during ZPP in main, sapi, win32, and Zend 2019-12-30 17:38:10 +01:00
Nikita Popov
1d6325fb01 Merge branch 'PHP-7.4'
* PHP-7.4:
  Fix leak when generator closed during yield in finally
2019-12-20 17:46:43 +01:00