archive/jdk
1
0
Fork 0
mirror of https://github.com/openjdk/jdk.git synced 2025-08-27 06:45:07 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

8304265: Implementation of Foreign Function and Memory API (Third Preview)

Browse source 
Co-authored-by: Maurizio Cimadamore <mcimadamore@openjdk.org>
Co-authored-by: Jorn Vernee <jvernee@openjdk.org>
Co-authored-by: Paul Sandoz <psandoz@openjdk.org>
Co-authored-by: Feilong Jiang <fjiang@openjdk.org>
Co-authored-by: Per Minborg <pminborg@openjdk.org>
Reviewed-by: erikj, jvernee, vlivanov, psandoz
This commit is contained in:
Per Minborg 2023-04-27 09:00:58 +00:00
parent 41d58533ac
commit cbccc4c817
267 changed files with 6947 additions and 8029 deletions
Download patch file Download diff file Expand all files Collapse all files

160
src/java.base/share/classes/java/lang/foreign/package-info.java
View file

@ -1,5 +1,5 @@
/*
* Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -31,77 +31,48 @@
*
* <p>
* The main abstraction introduced to support foreign memory access is {@link java.lang.foreign.MemorySegment}, which
* models a contiguous region of memory, residing either inside or outside the Java heap. The contents of a memory
* segment can be described using a {@link java.lang.foreign.MemoryLayout memory layout}, which provides
* basic operations to query sizes, offsets and alignment constraints. Memory layouts also provide
* an alternate, more abstract way, to <a href=MemorySegment.html#segment-deref>access memory segments</a>
* using {@linkplain java.lang.foreign.MemoryLayout#varHandle(java.lang.foreign.MemoryLayout.PathElement...) var handles},
* models a contiguous region of memory, residing either inside or outside the Java heap. Memory segments are
* typically allocated using an {@link java.lang.foreign.Arena}, which controls the lifetime of the regions of memory
* backing the segments it allocates. The contents of a memory segment can be described using a
* {@link java.lang.foreign.MemoryLayout memory layout}, which provides basic operations to query sizes, offsets and
* alignment constraints. Memory layouts also provide an alternate, more abstract way, to
* <a href=MemorySegment.html#segment-deref>access memory segments</a> using
* {@linkplain java.lang.foreign.MemoryLayout#varHandle(java.lang.foreign.MemoryLayout.PathElement...) var handles},
* which can be computed using <a href="MemoryLayout.html#layout-paths"><em>layout paths</em></a>.
*
* For example, to allocate an off-heap region of memory big enough to hold 10 values of the primitive type {@code int},
* and fill it with values ranging from {@code 0} to {@code 9}, we can use the following code:
*
* {@snippet lang = java:
* MemorySegment segment = MemorySegment.allocateNative(10 * 4, SegmentScope.auto());
* for (int i = 0 ; i < 10 ; i++) {
* segment.setAtIndex(ValueLayout.JAVA_INT, i, i);
* }
*}
*
* This code creates a <em>native</em> memory segment, that is, a memory segment backed by
* off-heap memory; the size of the segment is 40 bytes, enough to store 10 values of the primitive type {@code int}.
* The segment is associated with an {@linkplain java.lang.foreign.SegmentScope#auto() automatic scope}. This
* means that the off-heap region of memory backing the segment is managed, automatically, by the garbage collector.
* As such, the off-heap memory backing the native segment will be released at some unspecified
* point <em>after</em> the segment becomes <a href="../../../java/lang/ref/package.html#reachability">unreachable</a>.
* This is similar to what happens with direct buffers created via {@link java.nio.ByteBuffer#allocateDirect(int)}.
* It is also possible to manage the lifecycle of allocated native segments more directly, as shown in a later section.
* <p>
* Inside a loop, we then initialize the contents of the memory segment; note how the
* {@linkplain java.lang.foreign.MemorySegment#setAtIndex(ValueLayout.OfInt, long, int) access method}
* accepts a {@linkplain java.lang.foreign.ValueLayout value layout}, which specifies the size, alignment constraint,
* byte order as well as the Java type ({@code int}, in this case) associated with the access operation. More specifically,
* if we view the memory segment as a set of 10 adjacent slots, {@code s[i]}, where {@code 0 <= i < 10},
* where the size of each slot is exactly 4 bytes, the initialization logic above will set each slot
* so that {@code s[i] = i}, again where {@code 0 <= i < 10}.
*
* <h3 id="deallocation">Deterministic deallocation</h3>
*
* When writing code that manipulates memory segments, especially if backed by memory which resides outside the Java heap, it is
* often crucial that the resources associated with a memory segment are released when the segment is no longer in use,
* and in a timely fashion. For this reason, there might be cases where waiting for the garbage collector to determine that a segment
* is <a href="../../../java/lang/ref/package.html#reachability">unreachable</a> is not optimal.
* Clients that operate under these assumptions might want to programmatically release the memory backing a memory segment.
* This can be done, using the {@link java.lang.foreign.Arena} abstraction, as shown below:
*
* {@snippet lang = java:
* try (Arena arena = Arena.openConfined()) {
* try (Arena arena = Arena.ofConfined()) {
* MemorySegment segment = arena.allocate(10 * 4);
* for (int i = 0 ; i < 10 ; i++) {
* segment.setAtIndex(ValueLayout.JAVA_INT, i, i);
* }
* }
*}
* }
*
* This example is almost identical to the prior one; this time we first create an arena
* which is used to allocate multiple native segments which share the same life-cycle. That is, all the segments
* allocated by the arena will be associated with the same {@linkplain java.lang.foreign.SegmentScope scope}.
* Note the use of the <em>try-with-resources</em> construct: this idiom ensures that the off-heap region of memory backing the
* native segment will be released at the end of the block, according to the semantics described in Section {@jls 14.20.3}
* of <cite>The Java Language Specification</cite>.
*
* <h3 id="safety">Safety</h3>
*
* This API provides strong safety guarantees when it comes to memory access. First, when dereferencing a memory segment,
* This code creates a <em>native</em> memory segment, that is, a memory segment backed by
* off-heap memory; the size of the segment is 40 bytes, enough to store 10 values of the primitive type {@code int}.
* The native segment is allocated using a {@linkplain java.lang.foreign.Arena#ofConfined() confined arena}.
* As such, access to the native segment is restricted to the current thread (the thread that created the arena).
* Moreover, when the arena is closed, the native segment is invalidated, and its backing region of memory is
* deallocated. Note the use of the <em>try-with-resources</em> construct: this idiom ensures that the off-heap region
* of memory backing the native segment will be released at the end of the block, according to the semantics described
* in Section {@jls 14.20.3} of <cite>The Java Language Specification</cite>.
* <p>
* Memory segments provide strong safety guarantees when it comes to memory access. First, when accessing a memory segment,
* the access coordinates are validated (upon access), to make sure that access does not occur at any address which resides
* <em>outside</em> the boundaries of the memory segment used by the access operation. We call this guarantee <em>spatial safety</em>;
* in other words, access to memory segments is bounds-checked, in the same way as array access is, as described in
* Section {@jls 15.10.4} of <cite>The Java Language Specification</cite>.
* <p>
* Since memory segments created with an arena can become invalid (see above), segments are also validated (upon access) to make sure that
* the scope associated with the segment being accessed is still alive.
* We call this guarantee <em>temporal safety</em>. Together, spatial and temporal safety ensure that each memory access
* operation either succeeds - and accesses a valid location within the region of memory backing the memory segment - or fails.
* Additionally, to prevent a region of memory from being accessed <em>after</em> it has been deallocated
* (i.e. <em>use-after-free</em>), a segment is also validated (upon access) to make sure that the arena from which it
* has been obtained has not been closed. We call this guarantee <em>temporal safety</em>.
* <p>
* Together, spatial and temporal safety ensure that each memory access operation either succeeds - and accesses a valid
* location within the region of memory backing the memory segment - or fails.
*
* <h2 id="ffa">Foreign function access</h2>
* The key abstractions introduced to support foreign function access are {@link java.lang.foreign.SymbolLookup},
@ -111,7 +82,7 @@
* so that clients can perform foreign function calls directly in Java, without the need for intermediate layers of C/C++
* code (as is the case with the <a href="{@docRoot}/../specs/jni/index.html">Java Native Interface (JNI)</a>).
* <p>
* For example, to compute the length of a string using the C standard library function {@code strlen} on a Linux x64 platform,
* For example, to compute the length of a string using the C standard library function {@code strlen} on a Linux/x64 platform,
* we can use the following code:
*
* {@snippet lang = java:
@ -122,90 +93,39 @@
* FunctionDescriptor.of(ValueLayout.JAVA_LONG, ValueLayout.ADDRESS)
* );
*
* try (Arena arena = Arena.openConfined()) {
* try (Arena arena = Arena.ofConfined()) {
* MemorySegment cString = arena.allocateUtf8String("Hello");
* long len = (long)strlen.invoke(cString); // 5
* long len = (long)strlen.invokeExact(cString); // 5
* }
*}
*
* Here, we obtain a {@linkplain java.lang.foreign.Linker#nativeLinker() native linker} and we use it
* to {@linkplain java.lang.foreign.SymbolLookup#find(java.lang.String) look up} the {@code strlen} symbol in the
* standard C library; a <em>downcall method handle</em> targeting said symbol is subsequently
* to {@linkplain java.lang.foreign.SymbolLookup#find(java.lang.String) look up} the {@code strlen} function in the
* standard C library; a <em>downcall method handle</em> targeting said function is subsequently
* {@linkplain java.lang.foreign.Linker#downcallHandle(FunctionDescriptor, Linker.Option...) obtained}.
* To complete the linking successfully, we must provide a {@link java.lang.foreign.FunctionDescriptor} instance,
* describing the signature of the {@code strlen} function.
* From this information, the linker will uniquely determine the sequence of steps which will turn
* the method handle invocation (here performed using {@link java.lang.invoke.MethodHandle#invoke(java.lang.Object...)})
* the method handle invocation (here performed using {@link java.lang.invoke.MethodHandle#invokeExact(java.lang.Object...)})
* into a foreign function call, according to the rules specified by the ABI of the underlying platform.
* The {@link java.lang.foreign.Arena} class also provides many useful methods for
* interacting with foreign code, such as
* {@linkplain java.lang.foreign.SegmentAllocator#allocateUtf8String(java.lang.String) converting} Java strings into
* zero-terminated, UTF-8 strings, as demonstrated in the above example.
*
* <h3 id="upcalls">Upcalls</h3>
* The {@link java.lang.foreign.Linker} interface also allows clients to turn an existing method handle (which might point
* to a Java method) into a memory segment, so that Java code can effectively be passed to other foreign functions.
* For instance, we can write a method that compares two integer values, as follows:
*
* {@snippet lang=java :
* class IntComparator {
* static int intCompare(MemorySegment addr1, MemorySegment addr2) {
* return addr1.get(ValueLayout.JAVA_INT, 0) -
* addr2.get(ValueLayout.JAVA_INT, 0);
*
* }
* }
* }
*
* The above method accesses two foreign memory segments containing an integer value, and performs a simple comparison
* by returning the difference between such values. We can then obtain a method handle which targets the above static
* method, as follows:
*
* {@snippet lang = java:
* FunctionDescriptor intCompareDescriptor = FunctionDescriptor.of(ValueLayout.JAVA_INT,
* ValueLayout.ADDRESS.asUnbounded(),
* ValueLayout.ADDRESS.asUnbounded());
* MethodHandle intCompareHandle = MethodHandles.lookup().findStatic(IntComparator.class,
* "intCompare",
* intCompareDescriptor.toMethodType());
*}
*
* As before, we need to create a {@link java.lang.foreign.FunctionDescriptor} instance, this time describing the signature
* of the function pointer we want to create. The descriptor can be used to
* {@linkplain java.lang.foreign.FunctionDescriptor#toMethodType() derive} a method type
* that can be used to look up the method handle for {@code IntComparator.intCompare}.
* <p>
* Now that we have a method handle instance, we can turn it into a fresh function pointer,
* using the {@link java.lang.foreign.Linker} interface, as follows:
*
* {@snippet lang = java:
* SegmentScope scope = ...
* MemorySegment comparFunc = Linker.nativeLinker().upcallStub(
* intCompareHandle, intCompareDescriptor, scope);
* );
*}
*
* The {@link java.lang.foreign.FunctionDescriptor} instance created in the previous step is then used to
* {@linkplain java.lang.foreign.Linker#upcallStub(java.lang.invoke.MethodHandle, FunctionDescriptor, SegmentScope) create}
* a new upcall stub; the layouts in the function descriptors allow the linker to determine the sequence of steps which
* allow foreign code to call the stub for {@code intCompareHandle} according to the rules specified by the ABI of the
* underlying platform.
* The lifecycle of the upcall stub is tied to the {@linkplain java.lang.foreign.SegmentScope scope}
* provided when the upcall stub is created. This same scope is made available by the {@link java.lang.foreign.MemorySegment}
* instance returned by that method.
*
* <h2 id="restricted">Restricted methods</h2>
* Some methods in this package are considered <em>restricted</em>. Restricted methods are typically used to bind native
* foreign data and/or functions to first-class Java API elements which can then be used directly by clients. For instance
* the restricted method {@link java.lang.foreign.MemorySegment#ofAddress(long, long, SegmentScope)}
* can be used to create a fresh segment with the given spatial bounds out of a native address.
* the restricted method {@link java.lang.foreign.MemorySegment#reinterpret(long)} ()}
* can be used to create a fresh segment with the same address and temporal bounds,
* but with the provided size. This can be useful to resize memory segments obtained when interacting with native functions.
* <p>
* Binding foreign data and/or functions is generally unsafe and, if done incorrectly, can result in VM crashes,
* or memory corruption when the bound Java API element is accessed. For instance, in the case of
* {@link java.lang.foreign.MemorySegment#ofAddress(long, long, SegmentScope)}, if the provided spatial bounds are
* incorrect, a client of the segment returned by that method might crash the VM, or corrupt
* memory when attempting to access said segment. For these reasons, it is crucial for code that calls a restricted method
* to never pass arguments that might cause incorrect binding of foreign data and/or functions to a Java API.
* or memory corruption when the bound Java API element is accessed. For instance, incorrectly resizing a native
* memory sgement using {@link java.lang.foreign.MemorySegment#reinterpret(long)} can lead to a JVM crash, or, worse,
* lead to silent memory corruption when attempting to access the resized segment. For these reasons, it is crucial for
* code that calls a restricted method to never pass arguments that might cause incorrect binding of foreign data and/or
* functions to a Java API.
* <p>
* Given the potential danger of restricted methods, the Java runtime issues a warning on the standard error stream
* every time a restricted method is invoked. Such warnings can be disabled by granting access to restricted methods