diff --git a/.hgtags b/.hgtags index c24b50bd310..d213f1fbfe4 100644 --- a/.hgtags +++ b/.hgtags @@ -340,3 +340,5 @@ f242d4332f563648426a1b0fa02d8741beba19ef jdk9-b92 5ac6287ec71aafe021cc839d8bc828108d23aaba jdk-9+95 139f19d70350238e15e107945cea75082b6380b3 jdk-9+96 4edcff1b9a8875eb6380a2165dfec599e8e3f7c0 jdk-9+97 +d00ad2d9049ac60815f70bff445e95df85648bd2 jdk-9+98 +f9bcdce2df26678c3fe468130b535c0342c69b89 jdk-9+99 diff --git a/.hgtags-top-repo b/.hgtags-top-repo index 43b90b329b2..d10b68389a6 100644 --- a/.hgtags-top-repo +++ b/.hgtags-top-repo @@ -340,3 +340,5 @@ cf1dc4c035fb84693d4ae5ad818785cb4d1465d1 jdk9-b90 12a6fb4f070f8ca8fbca219ab9abf5da8908b317 jdk-9+95 5582a79892596169ebddb3e2c2aa44939e4e3f40 jdk-9+96 75c3897541ecb52ee16d001ea605b12971df7303 jdk-9+97 +48987460c7d49a29013963ee44d090194396bb61 jdk-9+98 +7c0577bea4c65d69c5bef67023a89d2efa4fb2f7 jdk-9+99 diff --git a/README b/README index e1fdec5d4ab..477b38887fc 100644 --- a/README +++ b/README @@ -6,7 +6,7 @@ README: The root repository can be obtained with something like: hg clone http://hg.openjdk.java.net/jdk9/jdk9 openjdk9 - + You can run the get_source.sh script located in the root repository to get the other needed repositories: cd openjdk9 && sh ./get_source.sh @@ -17,7 +17,7 @@ README: See http://openjdk.java.net/ for more information about OpenJDK. Simple Build Instructions: - + 0. Get the necessary system software/packages installed on your system, see http://hg.openjdk.java.net/jdk9/jdk9/raw-file/tip/README-builds.html @@ -28,10 +28,10 @@ Simple Build Instructions: 2. Configure the build: bash ./configure - + 3. Build the OpenJDK: make all - The resulting JDK image should be found in build/*/images/j2sdk-image + The resulting JDK image should be found in build/*/images/jdk where make is GNU make 3.81 or newer, /usr/bin/make on Linux usually is 3.81 or newer. Note that on Solaris, GNU make is called "gmake". diff --git a/README-builds.html b/README-builds.html index d81549d5ce4..42cc0f11b7c 100644 --- a/README-builds.html +++ b/README-builds.html @@ -250,9 +250,7 @@ Compilers, freetype, cups, and
  • Mac OS X

    -

    Install XCode 4.5.2 and also -install the "Command line tools" found under the preferences pane -"Downloads"

  • +

    Install XCode 6.3

    @@ -279,39 +277,67 @@ OpenJDK.

    Studio Compilers

    At a minimum, the Studio 12 Update 1 Compilers (containing -version 5.10 of the C and C++ compilers) is required, including specific +technetwork/server-storage/solarisstudio/downloads/index.htm">Studio 12 Update 4 Compilers (containing +version 5.13 of the C and C++ compilers) is required, including specific patches.

    -

    The Solaris SPARC patch list is:

    +

    The Solaris Studio installation should contain at least these packages:

    - +
    +

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PackageVersion
    developer/solarisstudio-124/backend12.4-1.0.6.0
    developer/solarisstudio-124/c++12.4-1.0.10.0
    developer/solarisstudio-124/cc12.4-1.0.4.0
    developer/solarisstudio-124/library/c++-libs12.4-1.0.10.0
    developer/solarisstudio-124/library/math-libs12.4-1.0.0.1
    developer/solarisstudio-124/library/studio-gccrt12.4-1.0.0.1
    developer/solarisstudio-124/studio-common12.4-1.0.0.1
    developer/solarisstudio-124/studio-ja12.4-1.0.0.1
    developer/solarisstudio-124/studio-legal12.4-1.0.0.1
    developer/solarisstudio-124/studio-zhCN12.4-1.0.0.1

    +
    -

    The Solaris X86 patch list is:

    - - +

    In particular backend 12.4-1.0.6.0 contains a critical patch for the sparc +version.

    Place the bin directory in PATH.

    @@ -1144,10 +1170,6 @@ where the resulting bits can be used.

    With Linux, it was just a matter of picking a stable distribution that is a good representative for Linux in general.

    -

    NOTE: We expect a change here from Fedora 9 to something else, but it has not -been completely determined yet, possibly Ubuntu 12.04 X64, unbiased community -feedback would be welcome on what a good choice would be here.

    -

    It is understood that most developers will NOT be using these specific versions, and in fact creating these specific versions may be difficult due to the age of some of this software. It is expected that developers are more often @@ -1176,7 +1198,7 @@ so that they can be dealt with accordingly.

    Linux X86 (32-bit) and X64 (64-bit) Oracle Enterprise Linux 6.4 - gcc 4.8.2 + gcc 4.9.2 JDK 8 2 or more 1 GB @@ -1184,8 +1206,8 @@ so that they can be dealt with accordingly.

    Solaris SPARCV9 (64-bit) - Solaris 10 Update 10 - Studio 12 Update 3 + patches + Solaris 11 Update 1 + Studio 12 Update 4 + patches JDK 8 4 or more 4 GB @@ -1193,8 +1215,8 @@ so that they can be dealt with accordingly.

    Solaris X64 (64-bit) - Solaris 10 Update 10 - Studio 12 Update 3 + patches + Solaris 11 Update 1 + Studio 12 Update 4 + patches JDK 8 4 or more 4 GB @@ -1221,7 +1243,7 @@ so that they can be dealt with accordingly.

    Mac OS X X64 (64-bit) Mac OS X 10.9 "Mavericks" - XCode 5.1.1 or newer + Xcode 6.3 or newer JDK 8 2 or more 4 GB diff --git a/README-builds.md b/README-builds.md index 21367477890..bddb467612e 100644 --- a/README-builds.md +++ b/README-builds.md @@ -215,9 +215,7 @@ And for specific systems: * **Mac OS X** - Install [XCode 4.5.2](https://developer.apple.com/xcode/) and also - install the "Command line tools" found under the preferences pane - "Downloads" + Install [XCode 6.3](https://developer.apple.com/xcode/) #### Linux @@ -239,36 +237,66 @@ OpenJDK. ##### Studio Compilers -At a minimum, the [Studio 12 Update 1 Compilers](http://www.oracle.com/ +At a minimum, the [Studio 12 Update 4 Compilers](http://www.oracle.com/ technetwork/server-storage/solarisstudio/downloads/index.htm) (containing -version 5.10 of the C and C++ compilers) is required, including specific +version 5.13 of the C and C++ compilers) is required, including specific patches. -The Solaris SPARC patch list is: +The Solaris Studio installation should contain at least these packages: - * 118683-05: SunOS 5.10: Patch for profiling libraries and assembler - * 119963-21: SunOS 5.10: Shared library patch for C++ - * 120753-08: SunOS 5.10: Microtasking libraries (libmtsk) patch - * 128228-09: Sun Studio 12 Update 1: Patch for Sun C++ Compiler - * 141860-03: Sun Studio 12 Update 1: Patch for Compiler Common patch for Sun C - C++ F77 F95 - * 141861-05: Sun Studio 12 Update 1: Patch for Sun C Compiler - * 142371-01: Sun Studio 12.1 Update 1: Patch for dbx - * 143384-02: Sun Studio 12 Update 1: Patch for debuginfo handling - * 143385-02: Sun Studio 12 Update 1: Patch for Compiler Common patch for Sun C - C++ F77 F95 - * 142369-01: Sun Studio 12.1: Patch for Performance Analyzer Tools +> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    **Package****Version**
    developer/solarisstudio-124/backend12.4-1.0.6.0
    developer/solarisstudio-124/c++12.4-1.0.10.0
    developer/solarisstudio-124/cc12.4-1.0.4.0
    developer/solarisstudio-124/library/c++-libs12.4-1.0.10.0
    developer/solarisstudio-124/library/math-libs12.4-1.0.0.1
    developer/solarisstudio-124/library/studio-gccrt12.4-1.0.0.1
    developer/solarisstudio-124/studio-common12.4-1.0.0.1
    developer/solarisstudio-124/studio-ja12.4-1.0.0.1
    developer/solarisstudio-124/studio-legal12.4-1.0.0.1
    developer/solarisstudio-124/studio-zhCN12.4-1.0.0.1
    -The Solaris X86 patch list is: - - * 119961-07: SunOS 5.10_x86, x64, Patch for profiling libraries and assembler - * 119964-21: SunOS 5.10_x86: Shared library patch for C++\_x86 - * 120754-08: SunOS 5.10_x86: Microtasking libraries (libmtsk) patch - * 141858-06: Sun Studio 12 Update 1_x86: Sun Compiler Common patch for x86 - backend - * 128229-09: Sun Studio 12 Update 1_x86: Patch for C++ Compiler - * 142363-05: Sun Studio 12 Update 1_x86: Patch for C Compiler - * 142368-01: Sun Studio 12.1_x86: Patch for Performance Analyzer Tools +In particular backend 12.4-1.0.6.0 contains a critical patch for the sparc +version. Place the `bin` directory in `PATH`. @@ -1044,10 +1072,6 @@ where the resulting bits can be used. With Linux, it was just a matter of picking a stable distribution that is a good representative for Linux in general. -**NOTE: We expect a change here from Fedora 9 to something else, but it has not -been completely determined yet, possibly Ubuntu 12.04 X64, unbiased community -feedback would be welcome on what a good choice would be here.** - It is understood that most developers will NOT be using these specific versions, and in fact creating these specific versions may be difficult due to the age of some of this software. It is expected that developers are more often @@ -1075,7 +1099,7 @@ so that they can be dealt with accordingly. Linux X86 (32-bit) and X64 (64-bit) Oracle Enterprise Linux 6.4 - gcc 4.8.2 + gcc 4.9.2 JDK 8 2 or more 1 GB @@ -1083,8 +1107,8 @@ so that they can be dealt with accordingly. Solaris SPARCV9 (64-bit) - Solaris 10 Update 10 - Studio 12 Update 3 + patches + Solaris 11 Update 1 + Studio 12 Update 4 + patches JDK 8 4 or more 4 GB @@ -1092,8 +1116,8 @@ so that they can be dealt with accordingly. Solaris X64 (64-bit) - Solaris 10 Update 10 - Studio 12 Update 3 + patches + Solaris 11 Update 1 + Studio 12 Update 4 + patches JDK 8 4 or more 4 GB @@ -1120,7 +1144,7 @@ so that they can be dealt with accordingly. Mac OS X X64 (64-bit) Mac OS X 10.9 "Mavericks" - XCode 5.1.1 or newer + Xcode 6.3 or newer JDK 8 2 or more 4 GB diff --git a/common/autoconf/build-performance.m4 b/common/autoconf/build-performance.m4 index 74f0ae6af9c..5c2f1b62642 100644 --- a/common/autoconf/build-performance.m4 +++ b/common/autoconf/build-performance.m4 @@ -1,5 +1,5 @@ # -# Copyright (c) 2011, 2012, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -149,6 +149,19 @@ AC_DEFUN_ONCE([BPERF_SETUP_BUILD_JOBS], AC_SUBST(JOBS) ]) +AC_DEFUN_ONCE([BPERF_SETUP_TEST_JOBS], +[ + # The number of test jobs will be chosen automatically if TEST_JOBS is 0 + AC_ARG_WITH(test-jobs, [AS_HELP_STRING([--with-test-jobs], + [number of parallel tests jobs to run @<:@based on build jobs@:>@])]) + if test "x$with_test_jobs" = x; then + TEST_JOBS=0 + else + TEST_JOBS=$with_test_jobs + fi + AC_SUBST(TEST_JOBS) +]) + AC_DEFUN([BPERF_SETUP_CCACHE], [ AC_ARG_ENABLE([ccache], diff --git a/common/autoconf/configure.ac b/common/autoconf/configure.ac index cafecd5a993..8e2823ba0c9 100644 --- a/common/autoconf/configure.ac +++ b/common/autoconf/configure.ac @@ -44,6 +44,7 @@ m4_include([boot-jdk.m4]) m4_include([build-performance.m4]) m4_include([flags.m4]) m4_include([help.m4]) +m4_include([hotspot.m4]) m4_include([jdk-options.m4]) m4_include([jdk-version.m4]) m4_include([libraries.m4]) @@ -94,9 +95,10 @@ JDKOPT_SETUP_OPEN_OR_CUSTOM # These are needed to be able to create a configuration name (and thus the output directory) JDKOPT_SETUP_JDK_VARIANT -JDKOPT_SETUP_JVM_INTERPRETER -JDKOPT_SETUP_JVM_VARIANTS +HOTSPOT_SETUP_JVM_INTERPRETER +HOTSPOT_SETUP_JVM_VARIANTS JDKOPT_SETUP_DEBUG_LEVEL +HOTSPOT_SETUP_DEBUG_LEVEL # With basic setup done, call the custom early hook. CUSTOM_EARLY_HOOK @@ -132,6 +134,7 @@ BASIC_SETUP_DEFAULT_MAKE_TARGET # We need build & target for this. JDKOPT_SETUP_JDK_OPTIONS +HOTSPOT_SETUP_HOTSPOT_OPTIONS JDKVER_SETUP_JDK_VERSION_NUMBERS ############################################################################### @@ -220,7 +223,7 @@ LIB_SETUP_LIBRARIES # ############################################################################### -JDKOPT_SETUP_BUILD_TWEAKS +HOTSPOT_SETUP_BUILD_TWEAKS JDKOPT_DETECT_INTREE_EC ############################################################################### @@ -233,6 +236,7 @@ JDKOPT_DETECT_INTREE_EC BPERF_SETUP_BUILD_CORES BPERF_SETUP_BUILD_MEMORY BPERF_SETUP_BUILD_JOBS +BPERF_SETUP_TEST_JOBS # Setup arguments for the boot jdk (after cores and memory have been setup) BOOTJDK_SETUP_BOOT_JDK_ARGUMENTS diff --git a/common/autoconf/flags.m4 b/common/autoconf/flags.m4 index 88f2e89dd6f..af375d28432 100644 --- a/common/autoconf/flags.m4 +++ b/common/autoconf/flags.m4 @@ -120,13 +120,17 @@ AC_DEFUN([FLAGS_SETUP_SYSROOT_FLAGS], AC_DEFUN_ONCE([FLAGS_SETUP_INIT_FLAGS], [ - # Option used to tell the compiler whether to create 32- or 64-bit executables + # COMPILER_TARGET_BITS_FLAG : option for selecting 32- or 64-bit output + # COMPILER_COMMAND_FILE_FLAG : option for passing a command file to the compiler if test "x$TOOLCHAIN_TYPE" = xxlc; then COMPILER_TARGET_BITS_FLAG="-q" + COMPILER_COMMAND_FILE_FLAG="-f" else COMPILER_TARGET_BITS_FLAG="-m" + COMPILER_COMMAND_FILE_FLAG="@" fi AC_SUBST(COMPILER_TARGET_BITS_FLAG) + AC_SUBST(COMPILER_COMMAND_FILE_FLAG) # FIXME: figure out if we should select AR flags depending on OS or toolchain. if test "x$OPENJDK_TARGET_OS" = xmacosx; then @@ -226,37 +230,38 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_LIBS], else SHARED_LIBRARY_FLAGS="-dynamiclib -compatibility_version 1.0.0 -current_version 1.0.0 $PICFLAG" fi - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker @loader_path/.' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,@loader_path/.' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -install_name -Xlinker @rpath/[$]1' + SET_SHARED_LIBRARY_NAME='-Wl,-install_name,@rpath/[$]1' SET_SHARED_LIBRARY_MAPFILE='' else # Default works for linux, might work on other platforms as well. SHARED_LIBRARY_FLAGS='-shared' - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker \$$$$ORIGIN[$]1' - SET_SHARED_LIBRARY_ORIGIN="-Xlinker -z -Xlinker origin $SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -soname=[$]1' - SET_SHARED_LIBRARY_MAPFILE='-Xlinker -version-script=[$]1' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,\$$$$ORIGIN[$]1' + SET_SHARED_LIBRARY_ORIGIN="-Wl,-z,origin $SET_EXECUTABLE_ORIGIN" + SET_SHARED_LIBRARY_NAME='-Wl,-soname=[$]1' + SET_SHARED_LIBRARY_MAPFILE='-Wl,-version-script=[$]1' fi elif test "x$TOOLCHAIN_TYPE" = xclang; then - PICFLAG='' C_FLAG_REORDER='' CXX_FLAG_REORDER='' if test "x$OPENJDK_TARGET_OS" = xmacosx; then # Linking is different on MacOSX + PICFLAG='' SHARED_LIBRARY_FLAGS="-dynamiclib -compatibility_version 1.0.0 -current_version 1.0.0 $PICFLAG" - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker @loader_path/.' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,@loader_path/.' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -install_name -Xlinker @rpath/[$]1' + SET_SHARED_LIBRARY_NAME='-Wl,-install_name,@rpath/[$]1' SET_SHARED_LIBRARY_MAPFILE='' else # Default works for linux, might work on other platforms as well. + PICFLAG='-fPIC' SHARED_LIBRARY_FLAGS='-shared' - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker \$$$$ORIGIN[$]1' - SET_SHARED_LIBRARY_ORIGIN="-Xlinker -z -Xlinker origin $SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -soname=[$]1' - SET_SHARED_LIBRARY_MAPFILE='-Xlinker -version-script=[$]1' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,\$$$$ORIGIN[$]1' + SET_SHARED_LIBRARY_ORIGIN="-Wl,-z,origin $SET_EXECUTABLE_ORIGIN" + SET_SHARED_LIBRARY_NAME='-Wl,-soname=[$]1' + SET_SHARED_LIBRARY_MAPFILE='-Wl,-version-script=[$]1' fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then PICFLAG="-KPIC" @@ -265,7 +270,7 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_LIBS], SHARED_LIBRARY_FLAGS="-G" SET_EXECUTABLE_ORIGIN='-R\$$$$ORIGIN[$]1' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='' + SET_SHARED_LIBRARY_NAME='-h [$]1' SET_SHARED_LIBRARY_MAPFILE='-M[$]1' elif test "x$TOOLCHAIN_TYPE" = xxlc; then PICFLAG="-qpic=large" @@ -280,7 +285,7 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_LIBS], PICFLAG="" C_FLAG_REORDER='' CXX_FLAG_REORDER='' - SHARED_LIBRARY_FLAGS="-LD" + SHARED_LIBRARY_FLAGS="-dll" SET_EXECUTABLE_ORIGIN='' SET_SHARED_LIBRARY_ORIGIN='' SET_SHARED_LIBRARY_NAME='' @@ -293,6 +298,7 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_LIBS], AC_SUBST(SET_SHARED_LIBRARY_ORIGIN) AC_SUBST(SET_SHARED_LIBRARY_NAME) AC_SUBST(SET_SHARED_LIBRARY_MAPFILE) + AC_SUBST(SHARED_LIBRARY_FLAGS) if test "x$OPENJDK_TARGET_OS" = xsolaris; then CFLAGS_JDK="${CFLAGS_JDK} -D__solaris__" @@ -573,6 +579,25 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_JDK], CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" ;; esac + elif test "x$TOOLCHAIN_TYPE" = xclang; then + if test "x$OPENJDK_TARGET_OS" = xlinux; then + if test "x$OPENJDK_TARGET_CPU" = xx86; then + # Force compatibility with i586 on 32 bit intel platforms. + COMMON_CCXXFLAGS="${COMMON_CCXXFLAGS} -march=i586" + fi + COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS $COMMON_CCXXFLAGS_JDK -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wformat=2 \ + -pipe -D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE" + case $OPENJDK_TARGET_CPU_ARCH in + ppc ) + # on ppc we don't prevent gcc to omit frame pointer but do prevent strict aliasing + CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" + ;; + * ) + COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS_JDK -fno-omit-frame-pointer" + CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" + ;; + esac + fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS $COMMON_CCXXFLAGS_JDK -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS" if test "x$OPENJDK_TARGET_CPU_ARCH" = xx86; then @@ -748,17 +773,17 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_JDK], # If this is a --hash-style=gnu system, use --hash-style=both, why? # We have previously set HAS_GNU_HASH if this is the case if test -n "$HAS_GNU_HASH"; then - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker --hash-style=both" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,--hash-style=both" fi if test "x$OPENJDK_TARGET_OS" = xlinux; then # And since we now know that the linker is gnu, then add -z defs, to forbid # undefined symbols in object files. - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker -z -Xlinker defs" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,-z,defs" case $DEBUG_LEVEL in release ) # tell linker to optimize libraries. # Should this be supplied to the OSS linker as well? - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker -O1" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,-O1" ;; slowdebug ) if test "x$HAS_LINKER_NOW" = "xtrue"; then @@ -785,7 +810,7 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_JDK], esac fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then - LDFLAGS_JDK="$LDFLAGS_JDK -z defs -xildoff -ztext" + LDFLAGS_JDK="$LDFLAGS_JDK -Wl,-z,defs -xildoff -ztext" LDFLAGS_CXX_JDK="$LDFLAGS_CXX_JDK -norunpath -xnolib" elif test "x$TOOLCHAIN_TYPE" = xxlc; then LDFLAGS_JDK="${LDFLAGS_JDK} -brtl -bnolibpath -bexpall -bernotok" @@ -803,17 +828,19 @@ AC_DEFUN_ONCE([FLAGS_SETUP_COMPILER_FLAGS_FOR_JDK], fi LDFLAGS_JDKEXE="${LDFLAGS_JDKEXE} /STACK:$LDFLAGS_STACK_SIZE" elif test "x$OPENJDK_TARGET_OS" = xlinux; then - LDFLAGS_JDKEXE="$LDFLAGS_JDKEXE -Xlinker --allow-shlib-undefined" + LDFLAGS_JDKEXE="$LDFLAGS_JDKEXE -Wl,--allow-shlib-undefined" fi # Customize LDFLAGS for libs LDFLAGS_JDKLIB="${LDFLAGS_JDK}" + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} ${SHARED_LIBRARY_FLAGS}" if test "x$TOOLCHAIN_TYPE" = xmicrosoft; then - LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} -dll -libpath:${OUTPUT_ROOT}/support/modules_libs/java.base" + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} \ + -libpath:${OUTPUT_ROOT}/support/modules_libs/java.base" JDKLIB_LIBS="" else - LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} ${SHARED_LIBRARY_FLAGS} \ + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} \ -L${OUTPUT_ROOT}/support/modules_libs/java.base${OPENJDK_TARGET_CPU_LIBDIR}" # On some platforms (mac) the linker warns about non existing -L dirs. diff --git a/common/autoconf/generated-configure.sh b/common/autoconf/generated-configure.sh index a322c533c01..f5e0859115c 100644 --- a/common/autoconf/generated-configure.sh +++ b/common/autoconf/generated-configure.sh @@ -646,11 +646,11 @@ JAVA_FLAGS_SMALL JAVA_FLAGS_JAVAC JAVA_FLAGS_BIG JAVA_FLAGS +TEST_JOBS JOBS MEMORY_SIZE NUM_CORES ENABLE_INTREE_EC -SALIB_NAME HOTSPOT_MAKE_ARGS LIBZIP_CAN_USE_MMAP LIBDL @@ -729,6 +729,7 @@ CXXFLAGS_DEBUG_SYMBOLS CFLAGS_DEBUG_SYMBOLS CXX_FLAG_DEPS C_FLAG_DEPS +SHARED_LIBRARY_FLAGS SET_SHARED_LIBRARY_MAPFILE SET_SHARED_LIBRARY_NAME SET_SHARED_LIBRARY_ORIGIN @@ -742,6 +743,7 @@ EXE_OUT_OPTION CC_OUT_OPTION STRIPFLAGS ARFLAGS +COMPILER_COMMAND_FILE_FLAG COMPILER_TARGET_BITS_FLAG JT_HOME JTREGEXE @@ -753,6 +755,7 @@ HOTSPOT_CXX HOTSPOT_RC HOTSPOT_MT BUILD_AS +BUILD_LDCXX BUILD_LD BUILD_AR BUILD_NM @@ -799,6 +802,7 @@ ac_ct_PROPER_COMPILER_CC PROPER_COMPILER_CC TOOLCHAIN_PATH_CC POTENTIAL_CC +TOOLCHAIN_VERSION VS_LIB VS_INCLUDE VS_PATH @@ -857,11 +861,11 @@ JDK_RC_PLATFORM_NAME PRODUCT_SUFFIX PRODUCT_NAME LAUNCHER_NAME +TEST_IN_BUILD COPYRIGHT_YEAR COMPRESS_JARS UNLIMITED_CRYPTO CACERTS_FILE -TEST_IN_BUILD BUILD_HEADLESS SUPPORT_HEADFUL SUPPORT_HEADLESS @@ -910,7 +914,6 @@ INCLUDE_SA JVM_VARIANT_CORE JVM_VARIANT_ZEROSHARK JVM_VARIANT_ZERO -JVM_VARIANT_KERNEL JVM_VARIANT_MINIMAL1 JVM_VARIANT_CLIENT JVM_VARIANT_SERVER @@ -1073,10 +1076,10 @@ with_conf_name with_output_sync with_default_make_target enable_headful -enable_hotspot_test_in_build with_cacerts_file enable_unlimited_crypto with_copyright_year +enable_hotspot_test_in_build with_milestone with_update_version with_user_release_suffix @@ -1142,6 +1145,7 @@ with_dxsdk_include with_num_cores with_memory_size with_jobs +with_test_jobs with_boot_jdk_jvmargs with_sjavac_server_java enable_sjavac @@ -1883,10 +1887,10 @@ Optional Features: --with-debug-level=fastdebug) [disabled] --disable-headful disable building headful support (graphical UI support) [enabled] - --enable-hotspot-test-in-build - run the Queens test after Hotspot build [disabled] --enable-unlimited-crypto Enable unlimited crypto policy [disabled] + --enable-hotspot-test-in-build + run the Queens test after Hotspot build [disabled] --enable-static-build enable static library build [disabled] --disable-warnings-as-errors do not consider native warnings to be an error @@ -1923,8 +1927,7 @@ Optional Packages: --with-jdk-variant JDK variant to build (normal) [normal] --with-jvm-interpreter JVM interpreter to build (template, cpp) [template] --with-jvm-variants JVM variants (separated by commas) to build (server, - client, minimal1, kernel, zero, zeroshark, core) - [server] + client, minimal1, zero, zeroshark, core) [server] --with-debug-level set the debug level (release, fastdebug, slowdebug, optimized (HotSpot build only)) [release] --with-devkit use this devkit for compilers, tools and resources @@ -2061,6 +2064,8 @@ Optional Packages: --with-memory-size=1024 [probed] --with-jobs number of parallel jobs to let make run [calculated based on cores and memory] + --with-test-jobs number of parallel tests jobs to run [based on build + jobs] --with-boot-jdk-jvmargs specify JVM arguments to be passed to all java invocations of boot JDK, overriding the default values, e.g --with-boot-jdk-jvmargs="-Xmx8G @@ -3747,7 +3752,7 @@ ac_configure="$SHELL $ac_aux_dir/configure" # Please don't use this var. # -# Copyright (c) 2011, 2012, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -3785,6 +3790,8 @@ ac_configure="$SHELL $ac_aux_dir/configure" # Please don't use this var. + + ################################################################################ # # Optionally enable distributed compilation of native code using icecc/icecream @@ -3939,7 +3946,11 @@ Then run configure with '--with-freetype-src='. This will automatically build the freetype library into '/lib64' for 64-bit builds or into '/lib32' for 32-bit builds. Afterwards you can always use '--with-freetype-include=/include' -and '--with-freetype-lib=/lib32|64' for other builds." +and '--with-freetype-lib=/lib32|64' for other builds. + +Alternatively you can unpack the sources like this to use the default directory: + +tar --one-top-level=$HOME/freetype --strip-components=1 -xzf freetype-2.5.3.tar.gz" ;; esac } @@ -4037,13 +4048,80 @@ pkgadd_help() { # questions. # +############################################################################### +# Check which interpreter of the JVM we want to build. +# Currently we have: +# template: Template interpreter (the default) +# cpp : C++ interpreter + + +############################################################################### +# Check which variants of the JVM that we want to build. +# Currently we have: +# server: normal interpreter and a C2 or tiered C1/C2 compiler +# client: normal interpreter and C1 (no C2 compiler) (only 32-bit platforms) +# minimal1: reduced form of client with optional VM services and features stripped out +# zero: no machine code interpreter, no compiler +# zeroshark: zero interpreter and shark/llvm compiler backend +# core: interpreter only, no compiler (only works on some platforms) + + + +############################################################################### +# Setup legacy vars/targets and new vars to deal with different debug levels. +# +# release: no debug information, all optimizations, no asserts. +# optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. +# fastdebug: debug information (-g), all optimizations, all asserts +# slowdebug: debug information (-g), no optimizations, all asserts +# +# +# Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. +# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. +# +# This code is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License version 2 only, as +# published by the Free Software Foundation. Oracle designates this +# particular file as subject to the "Classpath" exception as provided +# by Oracle in the LICENSE file that accompanied this code. +# +# This code is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# version 2 for more details (a copy is included in the LICENSE file that +# accompanied this code). +# +# You should have received a copy of the GNU General Public License version +# 2 along with this work; if not, write to the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +# +# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA +# or visit www.oracle.com if you need additional information or have any +# questions. +# +############################################################################### +# Check which variant of the JDK that we want to build. +# Currently we have: +# normal: standard edition +# but the custom make system may add other variants +# +# Effectively the JDK variant gives a name to a specific set of +# modules to compile into the JDK. + + +############################################################################### +# Set the debug level +# release: no debug information, all optimizations, no asserts. +# optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. +# fastdebug: debug information (-g), all optimizations, all asserts +# slowdebug: debug information (-g), no optimizations, all asserts ############################################################################### @@ -4054,8 +4132,6 @@ pkgadd_help() { - - ############################################################################### # # Enable or disable the elliptic curve crypto implementation @@ -4064,7 +4140,6 @@ pkgadd_help() { - ################################################################################ # # Gcov coverage data for hotspot @@ -4078,8 +4153,6 @@ pkgadd_help() { # - - # # Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. @@ -4728,7 +4801,7 @@ VS_SDK_PLATFORM_NAME_2013= #CUSTOM_AUTOCONF_INCLUDE # Do not change or remove the following line, it is needed for consistency checks: -DATE_WHEN_GENERATED=1449850507 +DATE_WHEN_GENERATED=1450277321 ############################################################################### # @@ -15607,17 +15680,6 @@ fi # These are needed to be able to create a configuration name (and thus the output directory) - ############################################################################### - # - # Check which variant of the JDK that we want to build. - # Currently we have: - # normal: standard edition - # but the custom make system may add other variants - # - # Effectively the JDK variant gives a name to a specific set of - # modules to compile into the JDK. In the future, these modules - # might even be Jigsaw modules. - # { $as_echo "$as_me:${as_lineno-$LINENO}: checking which variant of the JDK to build" >&5 $as_echo_n "checking which variant of the JDK to build... " >&6; } @@ -15639,14 +15701,6 @@ fi $as_echo "$JDK_VARIANT" >&6; } -############################################################################### -# -# Check which interpreter of the JVM we want to build. -# Currently we have: -# template: Template interpreter (the default) -# cpp : C++ interpreter -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking which interpreter of the JVM to build" >&5 -$as_echo_n "checking which interpreter of the JVM to build... " >&6; } # Check whether --with-jvm-interpreter was given. if test "${with_jvm_interpreter+set}" = set; then : @@ -15654,35 +15708,23 @@ if test "${with_jvm_interpreter+set}" = set; then : fi -if test "x$with_jvm_interpreter" = x; then - with_jvm_interpreter="template" -fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking which interpreter of the JVM to build" >&5 +$as_echo_n "checking which interpreter of the JVM to build... " >&6; } + if test "x$with_jvm_interpreter" = x; then + JVM_INTERPRETER="template" + else + JVM_INTERPRETER="$with_jvm_interpreter" + fi + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $JVM_INTERPRETER" >&5 +$as_echo "$JVM_INTERPRETER" >&6; } -JVM_INTERPRETER="$with_jvm_interpreter" - -if test "x$JVM_INTERPRETER" != xtemplate && test "x$JVM_INTERPRETER" != xcpp; then - as_fn_error $? "The available JVM interpreters are: template, cpp" "$LINENO" 5 -fi + if test "x$JVM_INTERPRETER" != xtemplate && test "x$JVM_INTERPRETER" != xcpp; then + as_fn_error $? "The available JVM interpreters are: template, cpp" "$LINENO" 5 + fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_jvm_interpreter" >&5 -$as_echo "$with_jvm_interpreter" >&6; } - - - ############################################################################### - # - # Check which variants of the JVM that we want to build. - # Currently we have: - # server: normal interpreter and a tiered C1/C2 compiler - # client: normal interpreter and C1 (no C2 compiler) (only 32-bit platforms) - # minimal1: reduced form of client with optional VM services and features stripped out - # kernel: kernel footprint JVM that passes the TCK without major performance problems, - # ie normal interpreter and C1, only the serial GC, kernel jvmti etc - # zero: no machine code interpreter, no compiler - # zeroshark: zero interpreter and shark/llvm compiler backend -# core: interpreter only, no compiler (only works on some platforms) { $as_echo "$as_me:${as_lineno-$LINENO}: checking which variants of the JVM to build" >&5 $as_echo_n "checking which variants of the JVM to build... " >&6; } @@ -15697,10 +15739,10 @@ fi fi JVM_VARIANTS=",$with_jvm_variants," - TEST_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,//' -e 's/client,//' -e 's/minimal1,//' -e 's/kernel,//' -e 's/zero,//' -e 's/zeroshark,//' -e 's/core,//'` + TEST_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,//' -e 's/client,//' -e 's/minimal1,//' -e 's/zero,//' -e 's/zeroshark,//' -e 's/core,//'` if test "x$TEST_VARIANTS" != "x,"; then - as_fn_error $? "The available JVM variants are: server, client, minimal1, kernel, zero, zeroshark, core" "$LINENO" 5 + as_fn_error $? "The available JVM variants are: server, client, minimal1, zero, zeroshark, core" "$LINENO" 5 fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_jvm_variants" >&5 $as_echo "$with_jvm_variants" >&6; } @@ -15708,7 +15750,6 @@ $as_echo "$with_jvm_variants" >&6; } JVM_VARIANT_SERVER=`$ECHO "$JVM_VARIANTS" | $SED -e '/,server,/!s/.*/false/g' -e '/,server,/s/.*/true/g'` JVM_VARIANT_CLIENT=`$ECHO "$JVM_VARIANTS" | $SED -e '/,client,/!s/.*/false/g' -e '/,client,/s/.*/true/g'` JVM_VARIANT_MINIMAL1=`$ECHO "$JVM_VARIANTS" | $SED -e '/,minimal1,/!s/.*/false/g' -e '/,minimal1,/s/.*/true/g'` - JVM_VARIANT_KERNEL=`$ECHO "$JVM_VARIANTS" | $SED -e '/,kernel,/!s/.*/false/g' -e '/,kernel,/s/.*/true/g'` JVM_VARIANT_ZERO=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zero,/!s/.*/false/g' -e '/,zero,/s/.*/true/g'` JVM_VARIANT_ZEROSHARK=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zeroshark,/!s/.*/false/g' -e '/,zeroshark,/s/.*/true/g'` JVM_VARIANT_CORE=`$ECHO "$JVM_VARIANTS" | $SED -e '/,core,/!s/.*/false/g' -e '/,core,/s/.*/true/g'` @@ -15718,11 +15759,6 @@ $as_echo "$with_jvm_variants" >&6; } as_fn_error $? "You cannot build a client JVM for a 64-bit machine." "$LINENO" 5 fi fi - if test "x$JVM_VARIANT_KERNEL" = xtrue; then - if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then - as_fn_error $? "You cannot build a kernel JVM for a 64-bit machine." "$LINENO" 5 - fi - fi if test "x$JVM_VARIANT_MINIMAL1" = xtrue; then if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then as_fn_error $? "You cannot build a minimal JVM for a 64-bit machine." "$LINENO" 5 @@ -15731,13 +15767,16 @@ $as_echo "$with_jvm_variants" >&6; } # Replace the commas with AND for use in the build directory name. ANDED_JVM_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/^,//' -e 's/,$//' -e 's/,/AND/g'` - COUNT_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,/1/' -e 's/client,/1/' -e 's/minimal1,/1/' -e 's/kernel,/1/' -e 's/zero,/1/' -e 's/zeroshark,/1/' -e 's/core,/1/'` + COUNT_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,/1/' -e 's/client,/1/' -e 's/minimal1,/1/' -e 's/zero,/1/' -e 's/zeroshark,/1/' -e 's/core,/1/'` if test "x$COUNT_VARIANTS" != "x,1"; then BUILDING_MULTIPLE_JVM_VARIANTS=yes else BUILDING_MULTIPLE_JVM_VARIANTS=no fi + if test "x$JVM_VARIANT_ZERO" = xtrue && test "x$BUILDING_MULTIPLE_JVM_VARIANTS" = xyes; then + as_fn_error $? "You cannot build multiple variants with zero." "$LINENO" 5 + fi @@ -15769,14 +15808,6 @@ $as_echo "$with_jvm_variants" >&6; } - ############################################################################### - # - # Set the debug level - # release: no debug information, all optimizations, no asserts. - # optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. - # fastdebug: debug information (-g), all optimizations, all asserts - # slowdebug: debug information (-g), no optimizations, all asserts - # DEBUG_LEVEL="release" { $as_echo "$as_me:${as_lineno-$LINENO}: checking which debug level to use" >&5 $as_echo_n "checking which debug level to use... " >&6; } @@ -15813,11 +15844,6 @@ $as_echo "$DEBUG_LEVEL" >&6; } fi - ############################################################################### - # - # Setup legacy vars/targets and new vars to deal with different debug levels. - # - case $DEBUG_LEVEL in release ) VARIANT="OPT" @@ -15887,10 +15913,6 @@ $as_echo "$DEBUG_LEVEL" >&6; } HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}minimal1 " fi - if test "x$JVM_VARIANT_KERNEL" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}kernel " - fi - if test "x$JVM_VARIANT_ZERO" = xtrue; then HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}zero " fi @@ -23122,12 +23144,8 @@ fi # We need build & target for this. - - ############################################################################### - # # Should we build a JDK/JVM with headful support (ie a graphical ui)? # We always build headless support. - # { $as_echo "$as_me:${as_lineno-$LINENO}: checking headful support" >&5 $as_echo_n "checking headful support... " >&6; } # Check whether --enable-headful was given. @@ -23159,25 +23177,7 @@ $as_echo "$headful_msg" >&6; } - # Control wether Hotspot runs Queens test after build. - # Check whether --enable-hotspot-test-in-build was given. -if test "${enable_hotspot_test_in_build+set}" = set; then : - enableval=$enable_hotspot_test_in_build; -else - enable_hotspot_test_in_build=no -fi - - if test "x$enable_hotspot_test_in_build" = "xyes"; then - TEST_IN_BUILD=true - else - TEST_IN_BUILD=false - fi - - - ############################################################################### - # # Choose cacerts source file - # # Check whether --with-cacerts-file was given. if test "${with_cacerts_file+set}" = set; then : @@ -23189,10 +23189,7 @@ fi fi - ############################################################################### - # # Enable or disable unlimited crypto - # # Check whether --enable-unlimited-crypto was given. if test "${enable_unlimited_crypto+set}" = set; then : enableval=$enable_unlimited_crypto; @@ -23207,10 +23204,7 @@ fi fi - ############################################################################### - # # Compress jars - # COMPRESS_JARS=false @@ -23232,6 +23226,22 @@ fi + # Control wether Hotspot runs Queens test after build. + # Check whether --enable-hotspot-test-in-build was given. +if test "${enable_hotspot_test_in_build+set}" = set; then : + enableval=$enable_hotspot_test_in_build; +else + enable_hotspot_test_in_build=no +fi + + if test "x$enable_hotspot_test_in_build" = "xyes"; then + TEST_IN_BUILD=true + else + TEST_IN_BUILD=false + fi + + + # Warn user that old version arguments are deprecated. @@ -31432,8 +31442,12 @@ $as_echo "$as_me: or run \"bash.exe -l\" from a VS command prompt and then run c # The microsoft toolchain also requires INCLUDE and LIB to be set. export INCLUDE="$VS_INCLUDE" export LIB="$VS_LIB" + else + # Currently we do not define this for other toolchains. This might change as the need arise. + TOOLCHAIN_VERSION= fi + # For solaris we really need solaris tools, and not the GNU equivalent. # The build tools on Solaris reside in /usr/ccs (C Compilation System), # so add that to path before starting to probe. @@ -45214,6 +45228,7 @@ $as_echo "$as_me: Rewriting BUILD_AR to \"$new_complete\"" >&6;} BUILD_AS="$BUILD_CC -c" # Just like for the target compiler, use the compiler as linker BUILD_LD="$BUILD_CC" + BUILD_LDCXX="$BUILD_CXX" PATH="$OLDPATH" else @@ -45222,6 +45237,7 @@ $as_echo "$as_me: Rewriting BUILD_AR to \"$new_complete\"" >&6;} BUILD_CC="$CC" BUILD_CXX="$CXX" BUILD_LD="$LD" + BUILD_LDCXX="$LDCXX" BUILD_NM="$NM" BUILD_AS="$AS" BUILD_SYSROOT_CFLAGS="$SYSROOT_CFLAGS" @@ -45239,6 +45255,7 @@ $as_echo "$as_me: Rewriting BUILD_AR to \"$new_complete\"" >&6;} + if test "x$TOOLCHAIN_TYPE" = xmicrosoft; then # For hotspot, we need these in Windows mixed path, # so rewrite them all. Need added .exe suffix. @@ -45398,7 +45415,7 @@ $as_echo "$supports" >&6; } # "-z relro" supported in GNU binutils 2.17 and later - LINKER_RELRO_FLAG="-Xlinker -z -Xlinker relro" + LINKER_RELRO_FLAG="-Wl,-z,relro" { $as_echo "$as_me:${as_lineno-$LINENO}: checking if linker supports \"$LINKER_RELRO_FLAG\"" >&5 $as_echo_n "checking if linker supports \"$LINKER_RELRO_FLAG\"... " >&6; } @@ -45448,7 +45465,7 @@ $as_echo "$supports" >&6; } # "-z now" supported in GNU binutils 2.11 and later - LINKER_NOW_FLAG="-Xlinker -z -Xlinker now" + LINKER_NOW_FLAG="-Wl,-z,now" { $as_echo "$as_me:${as_lineno-$LINENO}: checking if linker supports \"$LINKER_NOW_FLAG\"" >&5 $as_echo_n "checking if linker supports \"$LINKER_NOW_FLAG\"... " >&6; } @@ -45506,7 +45523,7 @@ $as_echo "$supports" >&6; } $as_echo_n "checking for broken SuSE 'ld' which only understands anonymous version tags in executables... " >&6; } $ECHO "SUNWprivate_1.1 { local: *; };" > version-script.map $ECHO "int main() { }" > main.c - if $CXX -Xlinker -version-script=version-script.map main.c 2>&5 >&5; then + if $CXX -Wl,-version-script=version-script.map main.c 2>&5 >&5; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } USING_BROKEN_SUSE_LD=no @@ -45905,14 +45922,18 @@ $as_echo "$tool_specified" >&6; } - # Option used to tell the compiler whether to create 32- or 64-bit executables + # COMPILER_TARGET_BITS_FLAG : option for selecting 32- or 64-bit output + # COMPILER_COMMAND_FILE_FLAG : option for passing a command file to the compiler if test "x$TOOLCHAIN_TYPE" = xxlc; then COMPILER_TARGET_BITS_FLAG="-q" + COMPILER_COMMAND_FILE_FLAG="-f" else COMPILER_TARGET_BITS_FLAG="-m" + COMPILER_COMMAND_FILE_FLAG="@" fi + # FIXME: figure out if we should select AR flags depending on OS or toolchain. if test "x$OPENJDK_TARGET_OS" = xmacosx; then ARFLAGS="-r" @@ -46646,37 +46667,38 @@ $as_echo "$ac_cv_c_bigendian" >&6; } else SHARED_LIBRARY_FLAGS="-dynamiclib -compatibility_version 1.0.0 -current_version 1.0.0 $PICFLAG" fi - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker @loader_path/.' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,@loader_path/.' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -install_name -Xlinker @rpath/$1' + SET_SHARED_LIBRARY_NAME='-Wl,-install_name,@rpath/$1' SET_SHARED_LIBRARY_MAPFILE='' else # Default works for linux, might work on other platforms as well. SHARED_LIBRARY_FLAGS='-shared' - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker \$$$$ORIGIN$1' - SET_SHARED_LIBRARY_ORIGIN="-Xlinker -z -Xlinker origin $SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -soname=$1' - SET_SHARED_LIBRARY_MAPFILE='-Xlinker -version-script=$1' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,\$$$$ORIGIN$1' + SET_SHARED_LIBRARY_ORIGIN="-Wl,-z,origin $SET_EXECUTABLE_ORIGIN" + SET_SHARED_LIBRARY_NAME='-Wl,-soname=$1' + SET_SHARED_LIBRARY_MAPFILE='-Wl,-version-script=$1' fi elif test "x$TOOLCHAIN_TYPE" = xclang; then - PICFLAG='' C_FLAG_REORDER='' CXX_FLAG_REORDER='' if test "x$OPENJDK_TARGET_OS" = xmacosx; then # Linking is different on MacOSX + PICFLAG='' SHARED_LIBRARY_FLAGS="-dynamiclib -compatibility_version 1.0.0 -current_version 1.0.0 $PICFLAG" - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker @loader_path/.' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,@loader_path/.' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -install_name -Xlinker @rpath/$1' + SET_SHARED_LIBRARY_NAME='-Wl,-install_name,@rpath/$1' SET_SHARED_LIBRARY_MAPFILE='' else # Default works for linux, might work on other platforms as well. + PICFLAG='-fPIC' SHARED_LIBRARY_FLAGS='-shared' - SET_EXECUTABLE_ORIGIN='-Xlinker -rpath -Xlinker \$$$$ORIGIN$1' - SET_SHARED_LIBRARY_ORIGIN="-Xlinker -z -Xlinker origin $SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='-Xlinker -soname=$1' - SET_SHARED_LIBRARY_MAPFILE='-Xlinker -version-script=$1' + SET_EXECUTABLE_ORIGIN='-Wl,-rpath,\$$$$ORIGIN$1' + SET_SHARED_LIBRARY_ORIGIN="-Wl,-z,origin $SET_EXECUTABLE_ORIGIN" + SET_SHARED_LIBRARY_NAME='-Wl,-soname=$1' + SET_SHARED_LIBRARY_MAPFILE='-Wl,-version-script=$1' fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then PICFLAG="-KPIC" @@ -46685,7 +46707,7 @@ $as_echo "$ac_cv_c_bigendian" >&6; } SHARED_LIBRARY_FLAGS="-G" SET_EXECUTABLE_ORIGIN='-R\$$$$ORIGIN$1' SET_SHARED_LIBRARY_ORIGIN="$SET_EXECUTABLE_ORIGIN" - SET_SHARED_LIBRARY_NAME='' + SET_SHARED_LIBRARY_NAME='-h $1' SET_SHARED_LIBRARY_MAPFILE='-M$1' elif test "x$TOOLCHAIN_TYPE" = xxlc; then PICFLAG="-qpic=large" @@ -46700,7 +46722,7 @@ $as_echo "$ac_cv_c_bigendian" >&6; } PICFLAG="" C_FLAG_REORDER='' CXX_FLAG_REORDER='' - SHARED_LIBRARY_FLAGS="-LD" + SHARED_LIBRARY_FLAGS="-dll" SET_EXECUTABLE_ORIGIN='' SET_SHARED_LIBRARY_ORIGIN='' SET_SHARED_LIBRARY_NAME='' @@ -46714,6 +46736,7 @@ $as_echo "$ac_cv_c_bigendian" >&6; } + if test "x$OPENJDK_TARGET_OS" = xsolaris; then CFLAGS_JDK="${CFLAGS_JDK} -D__solaris__" CXXFLAGS_JDK="${CXXFLAGS_JDK} -D__solaris__" @@ -47022,6 +47045,25 @@ $as_echo "$supports" >&6; } CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" ;; esac + elif test "x$TOOLCHAIN_TYPE" = xclang; then + if test "x$OPENJDK_TARGET_OS" = xlinux; then + if test "x$OPENJDK_TARGET_CPU" = xx86; then + # Force compatibility with i586 on 32 bit intel platforms. + COMMON_CCXXFLAGS="${COMMON_CCXXFLAGS} -march=i586" + fi + COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS $COMMON_CCXXFLAGS_JDK -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wformat=2 \ + -pipe -D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE" + case $OPENJDK_TARGET_CPU_ARCH in + ppc ) + # on ppc we don't prevent gcc to omit frame pointer but do prevent strict aliasing + CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" + ;; + * ) + COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS_JDK -fno-omit-frame-pointer" + CFLAGS_JDK="${CFLAGS_JDK} -fno-strict-aliasing" + ;; + esac + fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then COMMON_CCXXFLAGS_JDK="$COMMON_CCXXFLAGS $COMMON_CCXXFLAGS_JDK -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS" if test "x$OPENJDK_TARGET_CPU_ARCH" = xx86; then @@ -47197,17 +47239,17 @@ $as_echo "$supports" >&6; } # If this is a --hash-style=gnu system, use --hash-style=both, why? # We have previously set HAS_GNU_HASH if this is the case if test -n "$HAS_GNU_HASH"; then - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker --hash-style=both" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,--hash-style=both" fi if test "x$OPENJDK_TARGET_OS" = xlinux; then # And since we now know that the linker is gnu, then add -z defs, to forbid # undefined symbols in object files. - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker -z -Xlinker defs" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,-z,defs" case $DEBUG_LEVEL in release ) # tell linker to optimize libraries. # Should this be supplied to the OSS linker as well? - LDFLAGS_JDK="${LDFLAGS_JDK} -Xlinker -O1" + LDFLAGS_JDK="${LDFLAGS_JDK} -Wl,-O1" ;; slowdebug ) if test "x$HAS_LINKER_NOW" = "xtrue"; then @@ -47234,7 +47276,7 @@ $as_echo "$supports" >&6; } esac fi elif test "x$TOOLCHAIN_TYPE" = xsolstudio; then - LDFLAGS_JDK="$LDFLAGS_JDK -z defs -xildoff -ztext" + LDFLAGS_JDK="$LDFLAGS_JDK -Wl,-z,defs -xildoff -ztext" LDFLAGS_CXX_JDK="$LDFLAGS_CXX_JDK -norunpath -xnolib" elif test "x$TOOLCHAIN_TYPE" = xxlc; then LDFLAGS_JDK="${LDFLAGS_JDK} -brtl -bnolibpath -bexpall -bernotok" @@ -47252,17 +47294,19 @@ $as_echo "$supports" >&6; } fi LDFLAGS_JDKEXE="${LDFLAGS_JDKEXE} /STACK:$LDFLAGS_STACK_SIZE" elif test "x$OPENJDK_TARGET_OS" = xlinux; then - LDFLAGS_JDKEXE="$LDFLAGS_JDKEXE -Xlinker --allow-shlib-undefined" + LDFLAGS_JDKEXE="$LDFLAGS_JDKEXE -Wl,--allow-shlib-undefined" fi # Customize LDFLAGS for libs LDFLAGS_JDKLIB="${LDFLAGS_JDK}" + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} ${SHARED_LIBRARY_FLAGS}" if test "x$TOOLCHAIN_TYPE" = xmicrosoft; then - LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} -dll -libpath:${OUTPUT_ROOT}/support/modules_libs/java.base" + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} \ + -libpath:${OUTPUT_ROOT}/support/modules_libs/java.base" JDKLIB_LIBS="" else - LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} ${SHARED_LIBRARY_FLAGS} \ + LDFLAGS_JDKLIB="${LDFLAGS_JDKLIB} \ -L${OUTPUT_ROOT}/support/modules_libs/java.base${OPENJDK_TARGET_CPU_LIBDIR}" # On some platforms (mac) the linker warns about non existing -L dirs. @@ -47657,8 +47701,21 @@ $as_echo_n "checking what type of native debug symbols to use... " >&6; } # Check whether --with-native-debug-symbols was given. if test "${with_native_debug_symbols+set}" = set; then : withval=$with_native_debug_symbols; + if test "x$OPENJDK_TARGET_OS" = xaix; then + if test "x$withval" = xexternal || test "x$withval" = xzipped; then + as_fn_error $? "AIX only supports the parameters 'none' and 'internal' for --with-native-debug-symbols" "$LINENO" 5 + fi + fi + else - with_native_debug_symbols="zipped" + + if test "x$OPENJDK_TARGET_OS" = xaix; then + # AIX doesn't support 'zipped' so use 'internal' as default + with_native_debug_symbols="internal" + else + with_native_debug_symbols="zipped" + fi + fi NATIVE_DEBUG_SYMBOLS=$with_native_debug_symbols @@ -53467,6 +53524,1485 @@ $as_echo "$FREETYPE_LIB_PATH" >&6; } fi fi + if test "x$FOUND_FREETYPE" != xyes; then + FREETYPE_BASE_DIR="$HOME/freetype" + + windows_path="$FREETYPE_BASE_DIR" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + FREETYPE_BASE_DIR="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + FREETYPE_BASE_DIR="$unix_path" + fi + + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + + POTENTIAL_FREETYPE_INCLUDE_PATH="$FREETYPE_BASE_DIR/include" + POTENTIAL_FREETYPE_LIB_PATH="$FREETYPE_BASE_DIR/lib64" + METHOD="well-known location" + + # Let's start with an optimistic view of the world :-) + FOUND_FREETYPE=yes + + # First look for the canonical freetype main include file ft2build.h. + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Oh no! Let's try in the freetype2 directory. This is needed at least at Mac OS X Yosemite. + POTENTIAL_FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH/freetype2" + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Fail. + FOUND_FREETYPE=no + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + # Include file found, let's continue the sanity check. + { $as_echo "$as_me:${as_lineno-$LINENO}: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&5 +$as_echo "$as_me: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&6;} + + # Reset to default value + FREETYPE_BASE_NAME=freetype + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME"; then + if test "x$OPENJDK_TARGET_OS" = xmacosx \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH/${LIBRARY_PREFIX}freetype.6${SHARED_LIBRARY_SUFFIX}"; then + # On Mac OS X Yosemite, the symlink from libfreetype.dylib to libfreetype.6.dylib disappeared. Check + # for the .6 version explicitly. + FREETYPE_BASE_NAME=freetype.6 + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + { $as_echo "$as_me:${as_lineno-$LINENO}: Compensating for missing symlink by using version 6 explicitly" >&5 +$as_echo "$as_me: Compensating for missing symlink by using version 6 explicitly" >&6;} + else + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + else + if test "x$OPENJDK_TARGET_OS" = xwindows; then + # On Windows, we will need both .lib and .dll file. + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + elif test "x$OPENJDK_TARGET_OS" = xsolaris \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR/$FREETYPE_LIB_NAME"; then + # Found lib in isa dir, use that instead. + POTENTIAL_FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&5 +$as_echo "$as_me: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&6;} + fi + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_INCLUDE_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_INCLUDE_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_LIB_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_LIB_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_LIB_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_LIB_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_LIB_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype includes" >&5 +$as_echo_n "checking for freetype includes... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_INCLUDE_PATH" >&5 +$as_echo "$FREETYPE_INCLUDE_PATH" >&6; } + FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype libraries" >&5 +$as_echo_n "checking for freetype libraries... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_LIB_PATH" >&5 +$as_echo "$FREETYPE_LIB_PATH" >&6; } + fi + + else + + POTENTIAL_FREETYPE_INCLUDE_PATH="$FREETYPE_BASE_DIR/include" + POTENTIAL_FREETYPE_LIB_PATH="$FREETYPE_BASE_DIR/lib32" + METHOD="well-known location" + + # Let's start with an optimistic view of the world :-) + FOUND_FREETYPE=yes + + # First look for the canonical freetype main include file ft2build.h. + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Oh no! Let's try in the freetype2 directory. This is needed at least at Mac OS X Yosemite. + POTENTIAL_FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH/freetype2" + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Fail. + FOUND_FREETYPE=no + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + # Include file found, let's continue the sanity check. + { $as_echo "$as_me:${as_lineno-$LINENO}: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&5 +$as_echo "$as_me: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&6;} + + # Reset to default value + FREETYPE_BASE_NAME=freetype + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME"; then + if test "x$OPENJDK_TARGET_OS" = xmacosx \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH/${LIBRARY_PREFIX}freetype.6${SHARED_LIBRARY_SUFFIX}"; then + # On Mac OS X Yosemite, the symlink from libfreetype.dylib to libfreetype.6.dylib disappeared. Check + # for the .6 version explicitly. + FREETYPE_BASE_NAME=freetype.6 + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + { $as_echo "$as_me:${as_lineno-$LINENO}: Compensating for missing symlink by using version 6 explicitly" >&5 +$as_echo "$as_me: Compensating for missing symlink by using version 6 explicitly" >&6;} + else + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + else + if test "x$OPENJDK_TARGET_OS" = xwindows; then + # On Windows, we will need both .lib and .dll file. + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + elif test "x$OPENJDK_TARGET_OS" = xsolaris \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR/$FREETYPE_LIB_NAME"; then + # Found lib in isa dir, use that instead. + POTENTIAL_FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&5 +$as_echo "$as_me: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&6;} + fi + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_INCLUDE_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_INCLUDE_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_LIB_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_LIB_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_LIB_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_LIB_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_LIB_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype includes" >&5 +$as_echo_n "checking for freetype includes... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_INCLUDE_PATH" >&5 +$as_echo "$FREETYPE_INCLUDE_PATH" >&6; } + FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype libraries" >&5 +$as_echo_n "checking for freetype libraries... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_LIB_PATH" >&5 +$as_echo "$FREETYPE_LIB_PATH" >&6; } + fi + + fi + if test "x$FOUND_FREETYPE" != xyes && test -d $FREETYPE_BASE_DIR \ + && test -s "$FREETYPE_BASE_DIR/builds/windows/vc2010/freetype.vcxproj" && test "x$MSBUILD" != x; then + # Source is available, as a last resort try to build freetype in default location + + FREETYPE_SRC_PATH="$FREETYPE_BASE_DIR" + BUILD_FREETYPE=yes + + # Check if the freetype sources are acessible.. + if ! test -d $FREETYPE_SRC_PATH; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with-freetype-src specified, but can not find path \"$FREETYPE_SRC_PATH\" - ignoring --with-freetype-src" >&5 +$as_echo "$as_me: WARNING: --with-freetype-src specified, but can not find path \"$FREETYPE_SRC_PATH\" - ignoring --with-freetype-src" >&2;} + BUILD_FREETYPE=no + fi + # ..and contain a vc2010 project file + vcxproj_path="$FREETYPE_SRC_PATH/builds/windows/vc2010/freetype.vcxproj" + if test "x$BUILD_FREETYPE" = xyes && ! test -s $vcxproj_path; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Can not find project file $vcxproj_path (you may try a newer freetype version) - ignoring --with-freetype-src" >&5 +$as_echo "$as_me: WARNING: Can not find project file $vcxproj_path (you may try a newer freetype version) - ignoring --with-freetype-src" >&2;} + BUILD_FREETYPE=no + fi + # Now check if configure found a version of 'msbuild.exe' + if test "x$BUILD_FREETYPE" = xyes && test "x$MSBUILD" == x ; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Can not find an msbuild.exe executable (you may try to install .NET 4.0) - ignoring --with-freetype-src" >&5 +$as_echo "$as_me: WARNING: Can not find an msbuild.exe executable (you may try to install .NET 4.0) - ignoring --with-freetype-src" >&2;} + BUILD_FREETYPE=no + fi + + # Ready to go.. + if test "x$BUILD_FREETYPE" = xyes; then + # msbuild requires trailing slashes for output directories + freetype_lib_path="$FREETYPE_SRC_PATH/lib$OPENJDK_TARGET_CPU_BITS/" + freetype_lib_path_unix="$freetype_lib_path" + freetype_obj_path="$FREETYPE_SRC_PATH/obj$OPENJDK_TARGET_CPU_BITS/" + + unix_path="$vcxproj_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + windows_path=`$CYGPATH -m "$unix_path"` + vcxproj_path="$windows_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + windows_path=`cmd //c echo $unix_path` + vcxproj_path="$windows_path" + fi + + + unix_path="$freetype_lib_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + windows_path=`$CYGPATH -m "$unix_path"` + freetype_lib_path="$windows_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + windows_path=`cmd //c echo $unix_path` + freetype_lib_path="$windows_path" + fi + + + unix_path="$freetype_obj_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + windows_path=`$CYGPATH -m "$unix_path"` + freetype_obj_path="$windows_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + windows_path=`cmd //c echo $unix_path` + freetype_obj_path="$windows_path" + fi + + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + freetype_platform=x64 + else + freetype_platform=win32 + fi + + # The original freetype project file is for VS 2010 (i.e. 'v100'), + # so we have to adapt the toolset if building with any other toolsed (i.e. SDK). + # Currently 'PLATFORM_TOOLSET' is set in 'TOOLCHAIN_CHECK_POSSIBLE_VISUAL_STUDIO_ROOT'/ + # 'TOOLCHAIN_CHECK_POSSIBLE_WIN_SDK_ROOT' in toolchain_windows.m4 + { $as_echo "$as_me:${as_lineno-$LINENO}: Trying to compile freetype sources with PlatformToolset=$PLATFORM_TOOLSET to $freetype_lib_path_unix ..." >&5 +$as_echo "$as_me: Trying to compile freetype sources with PlatformToolset=$PLATFORM_TOOLSET to $freetype_lib_path_unix ..." >&6;} + + # First we try to build the freetype.dll + $ECHO -e "@echo off\n"\ + "$MSBUILD $vcxproj_path "\ + "/p:PlatformToolset=$PLATFORM_TOOLSET "\ + "/p:Configuration=\"Release Multithreaded\" "\ + "/p:Platform=$freetype_platform "\ + "/p:ConfigurationType=DynamicLibrary "\ + "/p:TargetName=freetype "\ + "/p:OutDir=\"$freetype_lib_path\" "\ + "/p:IntDir=\"$freetype_obj_path\" > freetype.log" > freetype.bat + cmd /c freetype.bat + + if test -s "$freetype_lib_path_unix/freetype.dll"; then + # If that succeeds we also build freetype.lib + $ECHO -e "@echo off\n"\ + "$MSBUILD $vcxproj_path "\ + "/p:PlatformToolset=$PLATFORM_TOOLSET "\ + "/p:Configuration=\"Release Multithreaded\" "\ + "/p:Platform=$freetype_platform "\ + "/p:ConfigurationType=StaticLibrary "\ + "/p:TargetName=freetype "\ + "/p:OutDir=\"$freetype_lib_path\" "\ + "/p:IntDir=\"$freetype_obj_path\" >> freetype.log" > freetype.bat + cmd /c freetype.bat + + if test -s "$freetype_lib_path_unix/freetype.lib"; then + # Once we build both, lib and dll, set freetype lib and include path appropriately + POTENTIAL_FREETYPE_INCLUDE_PATH="$FREETYPE_SRC_PATH/include" + POTENTIAL_FREETYPE_LIB_PATH="$freetype_lib_path_unix" + { $as_echo "$as_me:${as_lineno-$LINENO}: Compiling freetype sources succeeded! (see freetype.log for build results)" >&5 +$as_echo "$as_me: Compiling freetype sources succeeded! (see freetype.log for build results)" >&6;} + else + BUILD_FREETYPE=no + fi + else + BUILD_FREETYPE=no + fi + fi + + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + + POTENTIAL_FREETYPE_INCLUDE_PATH="$FREETYPE_BASE_DIR/include" + POTENTIAL_FREETYPE_LIB_PATH="$FREETYPE_BASE_DIR/lib64" + METHOD="well-known location" + + # Let's start with an optimistic view of the world :-) + FOUND_FREETYPE=yes + + # First look for the canonical freetype main include file ft2build.h. + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Oh no! Let's try in the freetype2 directory. This is needed at least at Mac OS X Yosemite. + POTENTIAL_FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH/freetype2" + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Fail. + FOUND_FREETYPE=no + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + # Include file found, let's continue the sanity check. + { $as_echo "$as_me:${as_lineno-$LINENO}: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&5 +$as_echo "$as_me: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&6;} + + # Reset to default value + FREETYPE_BASE_NAME=freetype + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME"; then + if test "x$OPENJDK_TARGET_OS" = xmacosx \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH/${LIBRARY_PREFIX}freetype.6${SHARED_LIBRARY_SUFFIX}"; then + # On Mac OS X Yosemite, the symlink from libfreetype.dylib to libfreetype.6.dylib disappeared. Check + # for the .6 version explicitly. + FREETYPE_BASE_NAME=freetype.6 + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + { $as_echo "$as_me:${as_lineno-$LINENO}: Compensating for missing symlink by using version 6 explicitly" >&5 +$as_echo "$as_me: Compensating for missing symlink by using version 6 explicitly" >&6;} + else + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + else + if test "x$OPENJDK_TARGET_OS" = xwindows; then + # On Windows, we will need both .lib and .dll file. + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + elif test "x$OPENJDK_TARGET_OS" = xsolaris \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR/$FREETYPE_LIB_NAME"; then + # Found lib in isa dir, use that instead. + POTENTIAL_FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&5 +$as_echo "$as_me: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&6;} + fi + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_INCLUDE_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_INCLUDE_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_LIB_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_LIB_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_LIB_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_LIB_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_LIB_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype includes" >&5 +$as_echo_n "checking for freetype includes... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_INCLUDE_PATH" >&5 +$as_echo "$FREETYPE_INCLUDE_PATH" >&6; } + FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype libraries" >&5 +$as_echo_n "checking for freetype libraries... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_LIB_PATH" >&5 +$as_echo "$FREETYPE_LIB_PATH" >&6; } + fi + + else + + POTENTIAL_FREETYPE_INCLUDE_PATH="$FREETYPE_BASE_DIR/include" + POTENTIAL_FREETYPE_LIB_PATH="$FREETYPE_BASE_DIR/lib32" + METHOD="well-known location" + + # Let's start with an optimistic view of the world :-) + FOUND_FREETYPE=yes + + # First look for the canonical freetype main include file ft2build.h. + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Oh no! Let's try in the freetype2 directory. This is needed at least at Mac OS X Yosemite. + POTENTIAL_FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH/freetype2" + if ! test -s "$POTENTIAL_FREETYPE_INCLUDE_PATH/ft2build.h"; then + # Fail. + FOUND_FREETYPE=no + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + # Include file found, let's continue the sanity check. + { $as_echo "$as_me:${as_lineno-$LINENO}: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&5 +$as_echo "$as_me: Found freetype include files at $POTENTIAL_FREETYPE_INCLUDE_PATH using $METHOD" >&6;} + + # Reset to default value + FREETYPE_BASE_NAME=freetype + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME"; then + if test "x$OPENJDK_TARGET_OS" = xmacosx \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH/${LIBRARY_PREFIX}freetype.6${SHARED_LIBRARY_SUFFIX}"; then + # On Mac OS X Yosemite, the symlink from libfreetype.dylib to libfreetype.6.dylib disappeared. Check + # for the .6 version explicitly. + FREETYPE_BASE_NAME=freetype.6 + FREETYPE_LIB_NAME="${LIBRARY_PREFIX}${FREETYPE_BASE_NAME}${SHARED_LIBRARY_SUFFIX}" + { $as_echo "$as_me:${as_lineno-$LINENO}: Compensating for missing symlink by using version 6 explicitly" >&5 +$as_echo "$as_me: Compensating for missing symlink by using version 6 explicitly" >&6;} + else + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/$FREETYPE_LIB_NAME. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + else + if test "x$OPENJDK_TARGET_OS" = xwindows; then + # On Windows, we will need both .lib and .dll file. + if ! test -s "$POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&5 +$as_echo "$as_me: Could not find $POTENTIAL_FREETYPE_LIB_PATH/${FREETYPE_BASE_NAME}.lib. Ignoring location." >&6;} + FOUND_FREETYPE=no + fi + elif test "x$OPENJDK_TARGET_OS" = xsolaris \ + && test -s "$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR/$FREETYPE_LIB_NAME"; then + # Found lib in isa dir, use that instead. + POTENTIAL_FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH$OPENJDK_TARGET_CPU_ISADIR" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&5 +$as_echo "$as_me: Rewriting to use $POTENTIAL_FREETYPE_LIB_PATH instead" >&6;} + fi + fi + fi + + if test "x$FOUND_FREETYPE" = xyes; then + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_INCLUDE_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_INCLUDE_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_INCLUDE_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_INCLUDE_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_INCLUDE_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_INCLUDE_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + # Only process if variable expands to non-empty + + if test "x$POTENTIAL_FREETYPE_LIB_PATH" != x; then + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + + # Input might be given as Windows format, start by converting to + # unix format. + path="$POTENTIAL_FREETYPE_LIB_PATH" + new_path=`$CYGPATH -u "$path"` + + # Cygwin tries to hide some aspects of the Windows file system, such that binaries are + # named .exe but called without that suffix. Therefore, "foo" and "foo.exe" are considered + # the same file, most of the time (as in "test -f"). But not when running cygpath -s, then + # "foo.exe" is OK but "foo" is an error. + # + # This test is therefore slightly more accurate than "test -f" to check for file precense. + # It is also a way to make sure we got the proper file name for the real test later on. + test_shortpath=`$CYGPATH -s -m "$new_path" 2> /dev/null` + if test "x$test_shortpath" = x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Cannot locate the the path of POTENTIAL_FREETYPE_LIB_PATH" "$LINENO" 5 + fi + + # Call helper function which possibly converts this using DOS-style short mode. + # If so, the updated path is stored in $new_path. + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-._/a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + shortmode_path=`$CYGPATH -s -m -a "$input_path"` + path_after_shortmode=`$CYGPATH -u "$shortmode_path"` + if test "x$path_after_shortmode" != "x$input_to_shortpath"; then + # Going to short mode and back again did indeed matter. Since short mode is + # case insensitive, let's make it lowercase to improve readability. + shortmode_path=`$ECHO "$shortmode_path" | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Now convert it back to Unix-style (cygpath) + input_path=`$CYGPATH -u "$shortmode_path"` + new_path="$input_path" + fi + fi + + test_cygdrive_prefix=`$ECHO $input_path | $GREP ^/cygdrive/` + if test "x$test_cygdrive_prefix" = x; then + # As a simple fix, exclude /usr/bin since it's not a real path. + if test "x`$ECHO $new_path | $GREP ^/usr/bin/`" = x; then + # The path is in a Cygwin special directory (e.g. /home). We need this converted to + # a path prefixed by /cygdrive for fixpath to work. + new_path="$CYGWIN_ROOT_PATH$input_path" + fi + fi + + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_colon=`$ECHO $path | $GREP ^.:` + new_path="$path" + if test "x$has_colon" = x; then + # Not in mixed or Windows style, start by that. + new_path=`cmd //c echo $path` + fi + + + input_path="$new_path" + # Check if we need to convert this using DOS-style short mode. If the path + # contains just simple characters, use it. Otherwise (spaces, weird characters), + # take no chances and rewrite it. + # Note: m4 eats our [], so we need to use [ and ] instead. + has_forbidden_chars=`$ECHO "$input_path" | $GREP [^-_/:a-zA-Z0-9]` + if test "x$has_forbidden_chars" != x; then + # Now convert it to mixed DOS-style, short mode (no spaces, and / instead of \) + new_path=`cmd /c "for %A in (\"$input_path\") do @echo %~sA"|$TR \\\\\\\\ / | $TR 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + fi + + + windows_path="$new_path" + if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.cygwin"; then + unix_path=`$CYGPATH -u "$windows_path"` + new_path="$unix_path" + elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then + unix_path=`$ECHO "$windows_path" | $SED -e 's,^\\(.\\):,/\\1,g' -e 's,\\\\,/,g'` + new_path="$unix_path" + fi + + if test "x$path" != "x$new_path"; then + POTENTIAL_FREETYPE_LIB_PATH="$new_path" + { $as_echo "$as_me:${as_lineno-$LINENO}: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&5 +$as_echo "$as_me: Rewriting POTENTIAL_FREETYPE_LIB_PATH to \"$new_path\"" >&6;} + fi + + # Save the first 10 bytes of this path to the storage, so fixpath can work. + all_fixpath_prefixes=("${all_fixpath_prefixes[@]}" "${new_path:0:10}") + + else + # We're on a unix platform. Hooray! :) + path="$POTENTIAL_FREETYPE_LIB_PATH" + has_space=`$ECHO "$path" | $GREP " "` + if test "x$has_space" != x; then + { $as_echo "$as_me:${as_lineno-$LINENO}: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&5 +$as_echo "$as_me: The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is invalid." >&6;} + as_fn_error $? "Spaces are not allowed in this path." "$LINENO" 5 + fi + + # Use eval to expand a potential ~ + eval path="$path" + if test ! -f "$path" && test ! -d "$path"; then + as_fn_error $? "The path of POTENTIAL_FREETYPE_LIB_PATH, which resolves as \"$path\", is not found." "$LINENO" 5 + fi + + if test -d "$path"; then + POTENTIAL_FREETYPE_LIB_PATH="`cd "$path"; $THEPWDCMD -L`" + else + dir="`$DIRNAME "$path"`" + base="`$BASENAME "$path"`" + POTENTIAL_FREETYPE_LIB_PATH="`cd "$dir"; $THEPWDCMD -L`/$base" + fi + fi + fi + + + FREETYPE_INCLUDE_PATH="$POTENTIAL_FREETYPE_INCLUDE_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype includes" >&5 +$as_echo_n "checking for freetype includes... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_INCLUDE_PATH" >&5 +$as_echo "$FREETYPE_INCLUDE_PATH" >&6; } + FREETYPE_LIB_PATH="$POTENTIAL_FREETYPE_LIB_PATH" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for freetype libraries" >&5 +$as_echo_n "checking for freetype libraries... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $FREETYPE_LIB_PATH" >&5 +$as_echo "$FREETYPE_LIB_PATH" >&6; } + fi + + fi + fi + fi else FREETYPE_BASE_DIR="$SYSROOT/usr" @@ -57085,13 +58621,6 @@ fi HOTSPOT_MAKE_ARGS="$HOTSPOT_TARGET" - # The name of the Service Agent jar. - SALIB_NAME="${LIBRARY_PREFIX}saproc${SHARED_LIBRARY_SUFFIX}" - if test "x$OPENJDK_TARGET_OS" = "xwindows"; then - SALIB_NAME="${LIBRARY_PREFIX}sawindbg${SHARED_LIBRARY_SUFFIX}" - fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if elliptic curve crypto implementation is present" >&5 $as_echo_n "checking if elliptic curve crypto implementation is present... " >&6; } @@ -57253,6 +58782,21 @@ $as_echo "$JOBS" >&6; } + # The number of test jobs will be chosen automatically if TEST_JOBS is 0 + +# Check whether --with-test-jobs was given. +if test "${with_test_jobs+set}" = set; then : + withval=$with_test_jobs; +fi + + if test "x$with_test_jobs" = x; then + TEST_JOBS=0 + else + TEST_JOBS=$with_test_jobs + fi + + + # Setup arguments for the boot jdk (after cores and memory have been setup) ############################################################################## diff --git a/common/autoconf/help.m4 b/common/autoconf/help.m4 index cf977f0598f..df5d0c87554 100644 --- a/common/autoconf/help.m4 +++ b/common/autoconf/help.m4 @@ -86,7 +86,11 @@ Then run configure with '--with-freetype-src='. This will automatically build the freetype library into '/lib64' for 64-bit builds or into '/lib32' for 32-bit builds. Afterwards you can always use '--with-freetype-include=/include' -and '--with-freetype-lib=/lib[32|64]' for other builds." +and '--with-freetype-lib=/lib[32|64]' for other builds. + +Alternatively you can unpack the sources like this to use the default directory: + +tar --one-top-level=$HOME/freetype --strip-components=1 -xzf freetype-2.5.3.tar.gz" ;; esac } diff --git a/common/autoconf/hotspot.m4 b/common/autoconf/hotspot.m4 new file mode 100644 index 00000000000..ec357f99572 --- /dev/null +++ b/common/autoconf/hotspot.m4 @@ -0,0 +1,268 @@ +# +# Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. +# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. +# +# This code is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License version 2 only, as +# published by the Free Software Foundation. Oracle designates this +# particular file as subject to the "Classpath" exception as provided +# by Oracle in the LICENSE file that accompanied this code. +# +# This code is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# version 2 for more details (a copy is included in the LICENSE file that +# accompanied this code). +# +# You should have received a copy of the GNU General Public License version +# 2 along with this work; if not, write to the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +# +# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA +# or visit www.oracle.com if you need additional information or have any +# questions. +# + +############################################################################### +# Check which interpreter of the JVM we want to build. +# Currently we have: +# template: Template interpreter (the default) +# cpp : C++ interpreter +AC_DEFUN_ONCE([HOTSPOT_SETUP_JVM_INTERPRETER], +[ + AC_ARG_WITH([jvm-interpreter], [AS_HELP_STRING([--with-jvm-interpreter], + [JVM interpreter to build (template, cpp) @<:@template@:>@])]) + + AC_MSG_CHECKING([which interpreter of the JVM to build]) + if test "x$with_jvm_interpreter" = x; then + JVM_INTERPRETER="template" + else + JVM_INTERPRETER="$with_jvm_interpreter" + fi + AC_MSG_RESULT([$JVM_INTERPRETER]) + + if test "x$JVM_INTERPRETER" != xtemplate && test "x$JVM_INTERPRETER" != xcpp; then + AC_MSG_ERROR([The available JVM interpreters are: template, cpp]) + fi + + AC_SUBST(JVM_INTERPRETER) +]) + +############################################################################### +# Check which variants of the JVM that we want to build. +# Currently we have: +# server: normal interpreter and a C2 or tiered C1/C2 compiler +# client: normal interpreter and C1 (no C2 compiler) (only 32-bit platforms) +# minimal1: reduced form of client with optional VM services and features stripped out +# zero: no machine code interpreter, no compiler +# zeroshark: zero interpreter and shark/llvm compiler backend +# core: interpreter only, no compiler (only works on some platforms) +AC_DEFUN_ONCE([HOTSPOT_SETUP_JVM_VARIANTS], +[ + AC_MSG_CHECKING([which variants of the JVM to build]) + AC_ARG_WITH([jvm-variants], [AS_HELP_STRING([--with-jvm-variants], + [JVM variants (separated by commas) to build (server, client, minimal1, zero, zeroshark, core) @<:@server@:>@])]) + + if test "x$with_jvm_variants" = x; then + with_jvm_variants="server" + fi + + JVM_VARIANTS=",$with_jvm_variants," + TEST_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,//' -e 's/client,//' -e 's/minimal1,//' -e 's/zero,//' -e 's/zeroshark,//' -e 's/core,//'` + + if test "x$TEST_VARIANTS" != "x,"; then + AC_MSG_ERROR([The available JVM variants are: server, client, minimal1, zero, zeroshark, core]) + fi + AC_MSG_RESULT([$with_jvm_variants]) + + JVM_VARIANT_SERVER=`$ECHO "$JVM_VARIANTS" | $SED -e '/,server,/!s/.*/false/g' -e '/,server,/s/.*/true/g'` + JVM_VARIANT_CLIENT=`$ECHO "$JVM_VARIANTS" | $SED -e '/,client,/!s/.*/false/g' -e '/,client,/s/.*/true/g'` + JVM_VARIANT_MINIMAL1=`$ECHO "$JVM_VARIANTS" | $SED -e '/,minimal1,/!s/.*/false/g' -e '/,minimal1,/s/.*/true/g'` + JVM_VARIANT_ZERO=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zero,/!s/.*/false/g' -e '/,zero,/s/.*/true/g'` + JVM_VARIANT_ZEROSHARK=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zeroshark,/!s/.*/false/g' -e '/,zeroshark,/s/.*/true/g'` + JVM_VARIANT_CORE=`$ECHO "$JVM_VARIANTS" | $SED -e '/,core,/!s/.*/false/g' -e '/,core,/s/.*/true/g'` + + if test "x$JVM_VARIANT_CLIENT" = xtrue; then + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + AC_MSG_ERROR([You cannot build a client JVM for a 64-bit machine.]) + fi + fi + if test "x$JVM_VARIANT_MINIMAL1" = xtrue; then + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + AC_MSG_ERROR([You cannot build a minimal JVM for a 64-bit machine.]) + fi + fi + + # Replace the commas with AND for use in the build directory name. + ANDED_JVM_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/^,//' -e 's/,$//' -e 's/,/AND/g'` + COUNT_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,/1/' -e 's/client,/1/' -e 's/minimal1,/1/' -e 's/zero,/1/' -e 's/zeroshark,/1/' -e 's/core,/1/'` + if test "x$COUNT_VARIANTS" != "x,1"; then + BUILDING_MULTIPLE_JVM_VARIANTS=yes + else + BUILDING_MULTIPLE_JVM_VARIANTS=no + fi + + if test "x$JVM_VARIANT_ZERO" = xtrue && test "x$BUILDING_MULTIPLE_JVM_VARIANTS" = xyes; then + AC_MSG_ERROR([You cannot build multiple variants with zero.]) + fi + + AC_SUBST(JVM_VARIANTS) + AC_SUBST(JVM_VARIANT_SERVER) + AC_SUBST(JVM_VARIANT_CLIENT) + AC_SUBST(JVM_VARIANT_MINIMAL1) + AC_SUBST(JVM_VARIANT_ZERO) + AC_SUBST(JVM_VARIANT_ZEROSHARK) + AC_SUBST(JVM_VARIANT_CORE) + + INCLUDE_SA=true + if test "x$JVM_VARIANT_ZERO" = xtrue ; then + INCLUDE_SA=false + fi + if test "x$JVM_VARIANT_ZEROSHARK" = xtrue ; then + INCLUDE_SA=false + fi + if test "x$OPENJDK_TARGET_OS" = xaix ; then + INCLUDE_SA=false + fi + if test "x$OPENJDK_TARGET_CPU" = xaarch64; then + INCLUDE_SA=false + fi + AC_SUBST(INCLUDE_SA) + + if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then + MACOSX_UNIVERSAL="true" + fi + + AC_SUBST(MACOSX_UNIVERSAL) +]) + + +############################################################################### +# Setup legacy vars/targets and new vars to deal with different debug levels. +# +# release: no debug information, all optimizations, no asserts. +# optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. +# fastdebug: debug information (-g), all optimizations, all asserts +# slowdebug: debug information (-g), no optimizations, all asserts +# +AC_DEFUN_ONCE([HOTSPOT_SETUP_DEBUG_LEVEL], +[ + case $DEBUG_LEVEL in + release ) + VARIANT="OPT" + FASTDEBUG="false" + DEBUG_CLASSFILES="false" + BUILD_VARIANT_RELEASE="" + HOTSPOT_DEBUG_LEVEL="product" + HOTSPOT_EXPORT="product" + ;; + fastdebug ) + VARIANT="DBG" + FASTDEBUG="true" + DEBUG_CLASSFILES="true" + BUILD_VARIANT_RELEASE="-fastdebug" + HOTSPOT_DEBUG_LEVEL="fastdebug" + HOTSPOT_EXPORT="fastdebug" + ;; + slowdebug ) + VARIANT="DBG" + FASTDEBUG="false" + DEBUG_CLASSFILES="true" + BUILD_VARIANT_RELEASE="-debug" + HOTSPOT_DEBUG_LEVEL="debug" + HOTSPOT_EXPORT="debug" + ;; + optimized ) + VARIANT="OPT" + FASTDEBUG="false" + DEBUG_CLASSFILES="false" + BUILD_VARIANT_RELEASE="-optimized" + HOTSPOT_DEBUG_LEVEL="optimized" + HOTSPOT_EXPORT="optimized" + ;; + esac + + # The debug level 'optimized' is a little special because it is currently only + # applicable to the HotSpot build where it means to build a completely + # optimized version of the VM without any debugging code (like for the + # 'release' debug level which is called 'product' in the HotSpot build) but + # with the exception that it can contain additional code which is otherwise + # protected by '#ifndef PRODUCT' macros. These 'optimized' builds are used to + # test new and/or experimental features which are not intended for customer + # shipment. Because these new features need to be tested and benchmarked in + # real world scenarios, we want to build the containing JDK at the 'release' + # debug level. + if test "x$DEBUG_LEVEL" = xoptimized; then + DEBUG_LEVEL="release" + fi + + ##### + # Generate the legacy makefile targets for hotspot. + # The hotspot api for selecting the build artifacts, really, needs to be improved. + # JDK-7195896 will fix this on the hotspot side by using the JVM_VARIANT_* variables to + # determine what needs to be built. All we will need to set here is all_product, all_fastdebug etc + # But until then ... + HOTSPOT_TARGET="" + + if test "x$JVM_VARIANT_SERVER" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL} " + fi + + if test "x$JVM_VARIANT_CLIENT" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}1 " + fi + + if test "x$JVM_VARIANT_MINIMAL1" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}minimal1 " + fi + + if test "x$JVM_VARIANT_ZERO" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}zero " + fi + + if test "x$JVM_VARIANT_ZEROSHARK" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}shark " + fi + + if test "x$JVM_VARIANT_CORE" = xtrue; then + HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}core " + fi + + HOTSPOT_TARGET="$HOTSPOT_TARGET docs export_$HOTSPOT_EXPORT" + + # On Macosx universal binaries are produced, but they only contain + # 64 bit intel. This invalidates control of which jvms are built + # from configure, but only server is valid anyway. Fix this + # when hotspot makefiles are rewritten. + if test "x$MACOSX_UNIVERSAL" = xtrue; then + HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} + fi + + ##### + + AC_SUBST(DEBUG_LEVEL) + AC_SUBST(VARIANT) + AC_SUBST(FASTDEBUG) + AC_SUBST(DEBUG_CLASSFILES) + AC_SUBST(BUILD_VARIANT_RELEASE) +]) + +AC_DEFUN_ONCE([HOTSPOT_SETUP_HOTSPOT_OPTIONS], +[ + # Control wether Hotspot runs Queens test after build. + AC_ARG_ENABLE([hotspot-test-in-build], [AS_HELP_STRING([--enable-hotspot-test-in-build], + [run the Queens test after Hotspot build @<:@disabled@:>@])],, + [enable_hotspot_test_in_build=no]) + if test "x$enable_hotspot_test_in_build" = "xyes"; then + TEST_IN_BUILD=true + else + TEST_IN_BUILD=false + fi + AC_SUBST(TEST_IN_BUILD) +]) + +AC_DEFUN_ONCE([HOTSPOT_SETUP_BUILD_TWEAKS], +[ + HOTSPOT_MAKE_ARGS="$HOTSPOT_TARGET" + AC_SUBST(HOTSPOT_MAKE_ARGS) +]) diff --git a/common/autoconf/jdk-options.m4 b/common/autoconf/jdk-options.m4 index 3f7a29e0abb..5f9ffb2e006 100644 --- a/common/autoconf/jdk-options.m4 +++ b/common/autoconf/jdk-options.m4 @@ -23,19 +23,16 @@ # questions. # +############################################################################### +# Check which variant of the JDK that we want to build. +# Currently we have: +# normal: standard edition +# but the custom make system may add other variants +# +# Effectively the JDK variant gives a name to a specific set of +# modules to compile into the JDK. AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_VARIANT], [ - ############################################################################### - # - # Check which variant of the JDK that we want to build. - # Currently we have: - # normal: standard edition - # but the custom make system may add other variants - # - # Effectively the JDK variant gives a name to a specific set of - # modules to compile into the JDK. In the future, these modules - # might even be Jigsaw modules. - # AC_MSG_CHECKING([which variant of the JDK to build]) AC_ARG_WITH([jdk-variant], [AS_HELP_STRING([--with-jdk-variant], [JDK variant to build (normal) @<:@normal@:>@])]) @@ -51,138 +48,14 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_VARIANT], AC_MSG_RESULT([$JDK_VARIANT]) ]) -AC_DEFUN_ONCE([JDKOPT_SETUP_JVM_INTERPRETER], -[ ############################################################################### -# -# Check which interpreter of the JVM we want to build. -# Currently we have: -# template: Template interpreter (the default) -# cpp : C++ interpreter -AC_MSG_CHECKING([which interpreter of the JVM to build]) -AC_ARG_WITH([jvm-interpreter], [AS_HELP_STRING([--with-jvm-interpreter], - [JVM interpreter to build (template, cpp) @<:@template@:>@])]) - -if test "x$with_jvm_interpreter" = x; then - with_jvm_interpreter="template" -fi - -JVM_INTERPRETER="$with_jvm_interpreter" - -if test "x$JVM_INTERPRETER" != xtemplate && test "x$JVM_INTERPRETER" != xcpp; then - AC_MSG_ERROR([The available JVM interpreters are: template, cpp]) -fi - -AC_SUBST(JVM_INTERPRETER) - -AC_MSG_RESULT([$with_jvm_interpreter]) -]) - -AC_DEFUN_ONCE([JDKOPT_SETUP_JVM_VARIANTS], -[ - - ############################################################################### - # - # Check which variants of the JVM that we want to build. - # Currently we have: - # server: normal interpreter and a tiered C1/C2 compiler - # client: normal interpreter and C1 (no C2 compiler) (only 32-bit platforms) - # minimal1: reduced form of client with optional VM services and features stripped out - # kernel: kernel footprint JVM that passes the TCK without major performance problems, - # ie normal interpreter and C1, only the serial GC, kernel jvmti etc - # zero: no machine code interpreter, no compiler - # zeroshark: zero interpreter and shark/llvm compiler backend -# core: interpreter only, no compiler (only works on some platforms) - AC_MSG_CHECKING([which variants of the JVM to build]) - AC_ARG_WITH([jvm-variants], [AS_HELP_STRING([--with-jvm-variants], - [JVM variants (separated by commas) to build (server, client, minimal1, kernel, zero, zeroshark, core) @<:@server@:>@])]) - - if test "x$with_jvm_variants" = x; then - with_jvm_variants="server" - fi - - JVM_VARIANTS=",$with_jvm_variants," - TEST_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,//' -e 's/client,//' -e 's/minimal1,//' -e 's/kernel,//' -e 's/zero,//' -e 's/zeroshark,//' -e 's/core,//'` - - if test "x$TEST_VARIANTS" != "x,"; then - AC_MSG_ERROR([The available JVM variants are: server, client, minimal1, kernel, zero, zeroshark, core]) - fi - AC_MSG_RESULT([$with_jvm_variants]) - - JVM_VARIANT_SERVER=`$ECHO "$JVM_VARIANTS" | $SED -e '/,server,/!s/.*/false/g' -e '/,server,/s/.*/true/g'` - JVM_VARIANT_CLIENT=`$ECHO "$JVM_VARIANTS" | $SED -e '/,client,/!s/.*/false/g' -e '/,client,/s/.*/true/g'` - JVM_VARIANT_MINIMAL1=`$ECHO "$JVM_VARIANTS" | $SED -e '/,minimal1,/!s/.*/false/g' -e '/,minimal1,/s/.*/true/g'` - JVM_VARIANT_KERNEL=`$ECHO "$JVM_VARIANTS" | $SED -e '/,kernel,/!s/.*/false/g' -e '/,kernel,/s/.*/true/g'` - JVM_VARIANT_ZERO=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zero,/!s/.*/false/g' -e '/,zero,/s/.*/true/g'` - JVM_VARIANT_ZEROSHARK=`$ECHO "$JVM_VARIANTS" | $SED -e '/,zeroshark,/!s/.*/false/g' -e '/,zeroshark,/s/.*/true/g'` - JVM_VARIANT_CORE=`$ECHO "$JVM_VARIANTS" | $SED -e '/,core,/!s/.*/false/g' -e '/,core,/s/.*/true/g'` - - if test "x$JVM_VARIANT_CLIENT" = xtrue; then - if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then - AC_MSG_ERROR([You cannot build a client JVM for a 64-bit machine.]) - fi - fi - if test "x$JVM_VARIANT_KERNEL" = xtrue; then - if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then - AC_MSG_ERROR([You cannot build a kernel JVM for a 64-bit machine.]) - fi - fi - if test "x$JVM_VARIANT_MINIMAL1" = xtrue; then - if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then - AC_MSG_ERROR([You cannot build a minimal JVM for a 64-bit machine.]) - fi - fi - - # Replace the commas with AND for use in the build directory name. - ANDED_JVM_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/^,//' -e 's/,$//' -e 's/,/AND/g'` - COUNT_VARIANTS=`$ECHO "$JVM_VARIANTS" | $SED -e 's/server,/1/' -e 's/client,/1/' -e 's/minimal1,/1/' -e 's/kernel,/1/' -e 's/zero,/1/' -e 's/zeroshark,/1/' -e 's/core,/1/'` - if test "x$COUNT_VARIANTS" != "x,1"; then - BUILDING_MULTIPLE_JVM_VARIANTS=yes - else - BUILDING_MULTIPLE_JVM_VARIANTS=no - fi - - AC_SUBST(JVM_VARIANTS) - AC_SUBST(JVM_VARIANT_SERVER) - AC_SUBST(JVM_VARIANT_CLIENT) - AC_SUBST(JVM_VARIANT_MINIMAL1) - AC_SUBST(JVM_VARIANT_KERNEL) - AC_SUBST(JVM_VARIANT_ZERO) - AC_SUBST(JVM_VARIANT_ZEROSHARK) - AC_SUBST(JVM_VARIANT_CORE) - - INCLUDE_SA=true - if test "x$JVM_VARIANT_ZERO" = xtrue ; then - INCLUDE_SA=false - fi - if test "x$JVM_VARIANT_ZEROSHARK" = xtrue ; then - INCLUDE_SA=false - fi - if test "x$OPENJDK_TARGET_OS" = xaix ; then - INCLUDE_SA=false - fi - if test "x$OPENJDK_TARGET_CPU" = xaarch64; then - INCLUDE_SA=false - fi - AC_SUBST(INCLUDE_SA) - - if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then - MACOSX_UNIVERSAL="true" - fi - - AC_SUBST(MACOSX_UNIVERSAL) -]) - +# Set the debug level +# release: no debug information, all optimizations, no asserts. +# optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. +# fastdebug: debug information (-g), all optimizations, all asserts +# slowdebug: debug information (-g), no optimizations, all asserts AC_DEFUN_ONCE([JDKOPT_SETUP_DEBUG_LEVEL], [ - ############################################################################### - # - # Set the debug level - # release: no debug information, all optimizations, no asserts. - # optimized: no debug information, all optimizations, no asserts, HotSpot target is 'optimized'. - # fastdebug: debug information (-g), all optimizations, all asserts - # slowdebug: debug information (-g), no optimizations, all asserts - # DEBUG_LEVEL="release" AC_MSG_CHECKING([which debug level to use]) AC_ARG_ENABLE([debug], [AS_HELP_STRING([--enable-debug], @@ -208,118 +81,8 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_DEBUG_LEVEL], test "x$DEBUG_LEVEL" != xslowdebug; then AC_MSG_ERROR([Allowed debug levels are: release, fastdebug and slowdebug]) fi - - - ############################################################################### - # - # Setup legacy vars/targets and new vars to deal with different debug levels. - # - - case $DEBUG_LEVEL in - release ) - VARIANT="OPT" - FASTDEBUG="false" - DEBUG_CLASSFILES="false" - BUILD_VARIANT_RELEASE="" - HOTSPOT_DEBUG_LEVEL="product" - HOTSPOT_EXPORT="product" - ;; - fastdebug ) - VARIANT="DBG" - FASTDEBUG="true" - DEBUG_CLASSFILES="true" - BUILD_VARIANT_RELEASE="-fastdebug" - HOTSPOT_DEBUG_LEVEL="fastdebug" - HOTSPOT_EXPORT="fastdebug" - ;; - slowdebug ) - VARIANT="DBG" - FASTDEBUG="false" - DEBUG_CLASSFILES="true" - BUILD_VARIANT_RELEASE="-debug" - HOTSPOT_DEBUG_LEVEL="debug" - HOTSPOT_EXPORT="debug" - ;; - optimized ) - VARIANT="OPT" - FASTDEBUG="false" - DEBUG_CLASSFILES="false" - BUILD_VARIANT_RELEASE="-optimized" - HOTSPOT_DEBUG_LEVEL="optimized" - HOTSPOT_EXPORT="optimized" - ;; - esac - - # The debug level 'optimized' is a little special because it is currently only - # applicable to the HotSpot build where it means to build a completely - # optimized version of the VM without any debugging code (like for the - # 'release' debug level which is called 'product' in the HotSpot build) but - # with the exception that it can contain additional code which is otherwise - # protected by '#ifndef PRODUCT' macros. These 'optimized' builds are used to - # test new and/or experimental features which are not intended for customer - # shipment. Because these new features need to be tested and benchmarked in - # real world scenarios, we want to build the containing JDK at the 'release' - # debug level. - if test "x$DEBUG_LEVEL" = xoptimized; then - DEBUG_LEVEL="release" - fi - - ##### - # Generate the legacy makefile targets for hotspot. - # The hotspot api for selecting the build artifacts, really, needs to be improved. - # JDK-7195896 will fix this on the hotspot side by using the JVM_VARIANT_* variables to - # determine what needs to be built. All we will need to set here is all_product, all_fastdebug etc - # But until then ... - HOTSPOT_TARGET="" - - if test "x$JVM_VARIANT_SERVER" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL} " - fi - - if test "x$JVM_VARIANT_CLIENT" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}1 " - fi - - if test "x$JVM_VARIANT_MINIMAL1" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}minimal1 " - fi - - if test "x$JVM_VARIANT_KERNEL" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}kernel " - fi - - if test "x$JVM_VARIANT_ZERO" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}zero " - fi - - if test "x$JVM_VARIANT_ZEROSHARK" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}shark " - fi - - if test "x$JVM_VARIANT_CORE" = xtrue; then - HOTSPOT_TARGET="$HOTSPOT_TARGET${HOTSPOT_DEBUG_LEVEL}core " - fi - - HOTSPOT_TARGET="$HOTSPOT_TARGET docs export_$HOTSPOT_EXPORT" - - # On Macosx universal binaries are produced, but they only contain - # 64 bit intel. This invalidates control of which jvms are built - # from configure, but only server is valid anyway. Fix this - # when hotspot makefiles are rewritten. - if test "x$MACOSX_UNIVERSAL" = xtrue; then - HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} - fi - - ##### - - AC_SUBST(DEBUG_LEVEL) - AC_SUBST(VARIANT) - AC_SUBST(FASTDEBUG) - AC_SUBST(DEBUG_CLASSFILES) - AC_SUBST(BUILD_VARIANT_RELEASE) ]) - ############################################################################### # # Should we build only OpenJDK even if closed sources are present? @@ -367,12 +130,8 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_OPEN_OR_CUSTOM], AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], [ - - ############################################################################### - # # Should we build a JDK/JVM with headful support (ie a graphical ui)? # We always build headless support. - # AC_MSG_CHECKING([headful support]) AC_ARG_ENABLE([headful], [AS_HELP_STRING([--disable-headful], [disable building headful support (graphical UI support) @<:@enabled@:>@])], @@ -398,21 +157,7 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], AC_SUBST(SUPPORT_HEADFUL) AC_SUBST(BUILD_HEADLESS) - # Control wether Hotspot runs Queens test after build. - AC_ARG_ENABLE([hotspot-test-in-build], [AS_HELP_STRING([--enable-hotspot-test-in-build], - [run the Queens test after Hotspot build @<:@disabled@:>@])],, - [enable_hotspot_test_in_build=no]) - if test "x$enable_hotspot_test_in_build" = "xyes"; then - TEST_IN_BUILD=true - else - TEST_IN_BUILD=false - fi - AC_SUBST(TEST_IN_BUILD) - - ############################################################################### - # # Choose cacerts source file - # AC_ARG_WITH(cacerts-file, [AS_HELP_STRING([--with-cacerts-file], [specify alternative cacerts file])]) if test "x$with_cacerts_file" != x; then @@ -420,10 +165,7 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], fi AC_SUBST(CACERTS_FILE) - ############################################################################### - # # Enable or disable unlimited crypto - # AC_ARG_ENABLE(unlimited-crypto, [AS_HELP_STRING([--enable-unlimited-crypto], [Enable unlimited crypto policy @<:@disabled@:>@])],, [enable_unlimited_crypto=no]) @@ -434,10 +176,7 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], fi AC_SUBST(UNLIMITED_CRYPTO) - ############################################################################### - # # Compress jars - # COMPRESS_JARS=false AC_SUBST(COMPRESS_JARS) @@ -455,19 +194,6 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS], AC_SUBST(COPYRIGHT_YEAR) ]) -AC_DEFUN_ONCE([JDKOPT_SETUP_BUILD_TWEAKS], -[ - HOTSPOT_MAKE_ARGS="$HOTSPOT_TARGET" - AC_SUBST(HOTSPOT_MAKE_ARGS) - - # The name of the Service Agent jar. - SALIB_NAME="${LIBRARY_PREFIX}saproc${SHARED_LIBRARY_SUFFIX}" - if test "x$OPENJDK_TARGET_OS" = "xwindows"; then - SALIB_NAME="${LIBRARY_PREFIX}sawindbg${SHARED_LIBRARY_SUFFIX}" - fi - AC_SUBST(SALIB_NAME) -]) - ############################################################################### # # Enable or disable the elliptic curve crypto implementation @@ -487,7 +213,6 @@ AC_DEFUN_ONCE([JDKOPT_DETECT_INTREE_EC], AC_SUBST(ENABLE_INTREE_EC) ]) - AC_DEFUN_ONCE([JDKOPT_SETUP_DEBUG_SYMBOLS], [ # @@ -498,8 +223,21 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_DEBUG_SYMBOLS], AC_ARG_WITH([native-debug-symbols], [AS_HELP_STRING([--with-native-debug-symbols], [set the native debug symbol configuration (none, internal, external, zipped) @<:@zipped@:>@])], - [], - [with_native_debug_symbols="zipped"]) + [ + if test "x$OPENJDK_TARGET_OS" = xaix; then + if test "x$withval" = xexternal || test "x$withval" = xzipped; then + AC_MSG_ERROR([AIX only supports the parameters 'none' and 'internal' for --with-native-debug-symbols]) + fi + fi + ], + [ + if test "x$OPENJDK_TARGET_OS" = xaix; then + # AIX doesn't support 'zipped' so use 'internal' as default + with_native_debug_symbols="internal" + else + with_native_debug_symbols="zipped" + fi + ]) NATIVE_DEBUG_SYMBOLS=$with_native_debug_symbols AC_MSG_RESULT([$NATIVE_DEBUG_SYMBOLS]) @@ -632,5 +370,3 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_STATIC_BUILD], AC_SUBST(STATIC_BUILD) ]) - - diff --git a/common/autoconf/lib-freetype.m4 b/common/autoconf/lib-freetype.m4 index 06753706bd2..2683fe23b6f 100644 --- a/common/autoconf/lib-freetype.m4 +++ b/common/autoconf/lib-freetype.m4 @@ -321,6 +321,25 @@ AC_DEFUN_ONCE([LIB_SETUP_FREETYPE], BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(FREETYPE_BASE_DIR) LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib], [well-known location]) fi + if test "x$FOUND_FREETYPE" != xyes; then + FREETYPE_BASE_DIR="$HOME/freetype" + BASIC_WINDOWS_REWRITE_AS_UNIX_PATH(FREETYPE_BASE_DIR) + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib64], [well-known location]) + else + LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib32], [well-known location]) + fi + if test "x$FOUND_FREETYPE" != xyes && test -d $FREETYPE_BASE_DIR \ + && test -s "$FREETYPE_BASE_DIR/builds/windows/vc2010/freetype.vcxproj" && test "x$MSBUILD" != x; then + # Source is available, as a last resort try to build freetype in default location + LIB_BUILD_FREETYPE($FREETYPE_BASE_DIR) + if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then + LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib64], [well-known location]) + else + LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib32], [well-known location]) + fi + fi + fi else FREETYPE_BASE_DIR="$SYSROOT/usr" LIB_CHECK_POTENTIAL_FREETYPE([$FREETYPE_BASE_DIR/include], [$FREETYPE_BASE_DIR/lib], [well-known location]) diff --git a/common/autoconf/spec.gmk.in b/common/autoconf/spec.gmk.in index 10b65f2592d..86b4761709a 100644 --- a/common/autoconf/spec.gmk.in +++ b/common/autoconf/spec.gmk.in @@ -204,13 +204,12 @@ SUPPORT_HEADLESS:=@SUPPORT_HEADLESS@ # These are the libjvms that we want to build. # The java launcher uses the default. -# The others can be selected by specifying -client -server -minimal1 -kernel -zero or -zeroshark +# The others can be selected by specifying -client -server -minimal1 -zero or -zeroshark # on the java launcher command line. JVM_VARIANTS:=@JVM_VARIANTS@ JVM_VARIANT_SERVER:=@JVM_VARIANT_SERVER@ JVM_VARIANT_CLIENT:=@JVM_VARIANT_CLIENT@ JVM_VARIANT_MINIMAL1:=@JVM_VARIANT_MINIMAL1@ -JVM_VARIANT_KERNEL:=@JVM_VARIANT_KERNEL@ JVM_VARIANT_ZERO:=@JVM_VARIANT_ZERO@ JVM_VARIANT_ZEROSHARK:=@JVM_VARIANT_ZEROSHARK@ JVM_VARIANT_CORE:=@JVM_VARIANT_CORE@ @@ -270,6 +269,7 @@ SJAVAC_SERVER_DIR=$(MAKESUPPORT_OUTPUTDIR)/javacservers # Number of parallel jobs to use for compilation JOBS?=@JOBS@ +TEST_JOBS?=@TEST_JOBS@ # Default make target DEFAULT_MAKE_TARGET:=@DEFAULT_MAKE_TARGET@ @@ -280,6 +280,8 @@ FREETYPE_BUNDLE_LIB_PATH=@FREETYPE_BUNDLE_LIB_PATH@ CUPS_CFLAGS:=@CUPS_CFLAGS@ ALSA_LIBS:=@ALSA_LIBS@ ALSA_CFLAGS:=@ALSA_CFLAGS@ +LIBFFI_LIBS:=@LIBFFI_LIBS@ +LIBFFI_CFLAGS:=@LIBFFI_CFLAGS@ PACKAGE_PATH=@PACKAGE_PATH@ @@ -300,11 +302,15 @@ MACOSX_VERSION_MIN=@MACOSX_VERSION_MIN@ # Toolchain type: gcc, clang, solstudio, lxc, microsoft... TOOLCHAIN_TYPE:=@TOOLCHAIN_TYPE@ +TOOLCHAIN_VERSION := @TOOLCHAIN_VERSION@ # Option used to tell the compiler whether to create 32- or 64-bit executables COMPILER_TARGET_BITS_FLAG:=@COMPILER_TARGET_BITS_FLAG@ COMPILER_SUPPORTS_TARGET_BITS_FLAG=@COMPILER_SUPPORTS_TARGET_BITS_FLAG@ +# Option used to pass a command file to the compiler +COMPILER_COMMAND_FILE_FLAG:=@COMPILER_COMMAND_FILE_FLAG@ + CC_OUT_OPTION:=@CC_OUT_OPTION@ EXE_OUT_OPTION:=@EXE_OUT_OPTION@ LD_OUT_OPTION:=@LD_OUT_OPTION@ @@ -388,6 +394,7 @@ LDFLAGS_TESTEXE:=@LDFLAGS_TESTEXE@ BUILD_CC:=@FIXPATH@ @BUILD_ICECC@ @BUILD_CC@ BUILD_CXX:=@FIXPATH@ @BUILD_ICECC@ @BUILD_CXX@ BUILD_LD:=@FIXPATH@ @BUILD_LD@ +BUILD_LDCXX:=@FIXPATH@ @BUILD_LDCXX@ BUILD_AS:=@FIXPATH@ @BUILD_AS@ BUILD_AR:=@FIXPATH@ @BUILD_AR@ BUILD_NM:=@FIXPATH@ @BUILD_NM@ @@ -433,6 +440,8 @@ COMPRESS_JARS=@COMPRESS_JARS@ # (Note absence of := assignment, because we do not want to evaluate the macro body here) SET_SHARED_LIBRARY_NAME=@SET_SHARED_LIBRARY_NAME@ +SHARED_LIBRARY_FLAGS=@SHARED_LIBRARY_FLAGS@ + # Set origin using the linker, ie use the relative path to the dependent library to find the dependees. # (Note absence of := assignment, because we do not want to evaluate the macro body here) SET_SHARED_LIBRARY_ORIGIN=@SET_SHARED_LIBRARY_ORIGIN@ @@ -650,9 +659,6 @@ PNG_CFLAGS:=@PNG_CFLAGS@ # Misc # -# Name of Service Agent library -SALIB_NAME=@SALIB_NAME@ - INCLUDE_SA=@INCLUDE_SA@ OS_VERSION_MAJOR:=@OS_VERSION_MAJOR@ diff --git a/common/autoconf/toolchain.m4 b/common/autoconf/toolchain.m4 index 0d9fb7cbedd..3f5750dcc89 100644 --- a/common/autoconf/toolchain.m4 +++ b/common/autoconf/toolchain.m4 @@ -216,7 +216,11 @@ AC_DEFUN_ONCE([TOOLCHAIN_PRE_DETECTION], # The microsoft toolchain also requires INCLUDE and LIB to be set. export INCLUDE="$VS_INCLUDE" export LIB="$VS_LIB" + else + # Currently we do not define this for other toolchains. This might change as the need arise. + TOOLCHAIN_VERSION= fi + AC_SUBST(TOOLCHAIN_VERSION) # For solaris we really need solaris tools, and not the GNU equivalent. # The build tools on Solaris reside in /usr/ccs (C Compilation System), @@ -731,6 +735,7 @@ AC_DEFUN_ONCE([TOOLCHAIN_SETUP_BUILD_COMPILERS], BUILD_AS="$BUILD_CC -c" # Just like for the target compiler, use the compiler as linker BUILD_LD="$BUILD_CC" + BUILD_LDCXX="$BUILD_CXX" PATH="$OLDPATH" else @@ -739,6 +744,7 @@ AC_DEFUN_ONCE([TOOLCHAIN_SETUP_BUILD_COMPILERS], BUILD_CC="$CC" BUILD_CXX="$CXX" BUILD_LD="$LD" + BUILD_LDCXX="$LDCXX" BUILD_NM="$NM" BUILD_AS="$AS" BUILD_SYSROOT_CFLAGS="$SYSROOT_CFLAGS" @@ -749,6 +755,7 @@ AC_DEFUN_ONCE([TOOLCHAIN_SETUP_BUILD_COMPILERS], AC_SUBST(BUILD_CC) AC_SUBST(BUILD_CXX) AC_SUBST(BUILD_LD) + AC_SUBST(BUILD_LDCXX) AC_SUBST(BUILD_NM) AC_SUBST(BUILD_AS) AC_SUBST(BUILD_SYSROOT_CFLAGS) @@ -822,13 +829,13 @@ AC_DEFUN_ONCE([TOOLCHAIN_MISC_CHECKS], [HAS_CFLAG_OPTIMIZE_DEBUG=false]) # "-z relro" supported in GNU binutils 2.17 and later - LINKER_RELRO_FLAG="-Xlinker -z -Xlinker relro" + LINKER_RELRO_FLAG="-Wl,-z,relro" FLAGS_LINKER_CHECK_ARGUMENTS([$LINKER_RELRO_FLAG], [HAS_LINKER_RELRO=true], [HAS_LINKER_RELRO=false]) # "-z now" supported in GNU binutils 2.11 and later - LINKER_NOW_FLAG="-Xlinker -z -Xlinker now" + LINKER_NOW_FLAG="-Wl,-z,now" FLAGS_LINKER_CHECK_ARGUMENTS([$LINKER_NOW_FLAG], [HAS_LINKER_NOW=true], [HAS_LINKER_NOW=false]) @@ -841,7 +848,7 @@ AC_DEFUN_ONCE([TOOLCHAIN_MISC_CHECKS], AC_MSG_CHECKING([for broken SuSE 'ld' which only understands anonymous version tags in executables]) $ECHO "SUNWprivate_1.1 { local: *; };" > version-script.map $ECHO "int main() { }" > main.c - if $CXX -Xlinker -version-script=version-script.map main.c 2>&AS_MESSAGE_LOG_FD >&AS_MESSAGE_LOG_FD; then + if $CXX -Wl,-version-script=version-script.map main.c 2>&AS_MESSAGE_LOG_FD >&AS_MESSAGE_LOG_FD; then AC_MSG_RESULT(no) USING_BROKEN_SUSE_LD=no else diff --git a/common/bin/compare.sh b/common/bin/compare.sh index 0c63bdcbeca..9e680f08b0a 100644 --- a/common/bin/compare.sh +++ b/common/bin/compare.sh @@ -37,13 +37,18 @@ fi if [ "$OPENJDK_TARGET_OS" = "macosx" ]; then FULLDUMP_CMD="$OTOOL -v -V -h -X -d" LDD_CMD="$OTOOL -L" - DIS_CMD="$OTOOL -v -t" + DIS_CMD="$OTOOL -v -V -t" STAT_PRINT_SIZE="-f %z" elif [ "$OPENJDK_TARGET_OS" = "windows" ]; then FULLDUMP_CMD="$DUMPBIN -all" LDD_CMD="$DUMPBIN -dependants | $GREP .dll" DIS_CMD="$DUMPBIN -disasm:nobytes" STAT_PRINT_SIZE="-c %s" +elif [ "$OPENJDK_TARGET_OS" = "aix" ]; then + FULLDUMP_CMD="dump -h -r -t -n -X64" + LDD_CMD="$LDD" + DIS_CMD="$OBJDUMP -d" + STAT_PRINT_SIZE="-c %s" else FULLDUMP_CMD="$READELF -a" LDD_CMD="$LDD" @@ -730,6 +735,9 @@ compare_bin_file() { # Some symbols get seemingly random 15 character prefixes. Filter them out. $NM -a $ORIG_OTHER_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SED 's/^\([a-zA-Z] [\.\$]\)[a-zA-Z0-9_\$]\{15,15\}\./\1./g' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.other $NM -a $ORIG_THIS_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SED 's/^\([a-zA-Z] [\.\$]\)[a-zA-Z0-9_\$]\{15,15\}\./\1./g' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.this + elif [ "$OPENJDK_TARGET_OS" = "aix" ]; then + $OBJDUMP -T $ORIG_OTHER_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.other + $OBJDUMP -T $ORIG_THIS_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.this else $NM -a $ORIG_OTHER_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.other $NM -a $ORIG_THIS_FILE 2> /dev/null | $GREP -v $NAME | $AWK '{print $2, $3, $4, $5}' | $SYM_SORT_CMD > $WORK_FILE_BASE.symbols.this @@ -796,14 +804,21 @@ compare_bin_file() { DEP_MSG=" - " fi + # Some linux compilers add a unique Build ID + if [ "$OPENJDK_TARGET_OS" = "linux" ]; then + BUILD_ID_FILTER="$SED -r 's/(Build ID:) [0-9a-f]{40}/\1/'" + else + BUILD_ID_FILTER="$CAT" + fi + # Compare fulldump output if [ -n "$FULLDUMP_CMD" ] && [ -z "$SKIP_FULLDUMP_DIFF" ]; then if [ -z "$FULLDUMP_DIFF_FILTER" ]; then FULLDUMP_DIFF_FILTER="$CAT" fi - $FULLDUMP_CMD $OTHER_FILE | eval "$FULLDUMP_DIFF_FILTER" \ + $FULLDUMP_CMD $OTHER_FILE | eval "$BUILD_ID_FILTER" | eval "$FULLDUMP_DIFF_FILTER" \ > $WORK_FILE_BASE.fulldump.other 2>&1 - $FULLDUMP_CMD $THIS_FILE | eval "$FULLDUMP_DIFF_FILTER" \ + $FULLDUMP_CMD $THIS_FILE | eval "$BUILD_ID_FILTER" | eval "$FULLDUMP_DIFF_FILTER" \ > $WORK_FILE_BASE.fulldump.this 2>&1 LC_ALL=C $DIFF $WORK_FILE_BASE.fulldump.other $WORK_FILE_BASE.fulldump.this \ diff --git a/common/bin/compare_exceptions.sh.incl b/common/bin/compare_exceptions.sh.incl index 79ef2f2c6d9..1ee38e29021 100644 --- a/common/bin/compare_exceptions.sh.incl +++ b/common/bin/compare_exceptions.sh.incl @@ -57,14 +57,18 @@ ACCEPTED_BIN_DIFF=" ./demo/jvmti/mtrace/lib/libmtrace.so ./demo/jvmti/versionCheck/lib/libversionCheck.so ./demo/jvmti/waiters/lib/libwaiters.so +./lib/i386/client/libjsig.so ./lib/i386/client/libjvm.so ./lib/i386/libattach.so ./lib/i386/libdt_socket.so ./lib/i386/libinstrument.so ./lib/i386/libjsdt.so +./lib/i386/libjsig.so ./lib/i386/libmanagement.so +./lib/i386/libnet.so ./lib/i386/libnpt.so ./lib/i386/libverify.so +./lib/i386/server/libjsig.so ./lib/i386/server/libjvm.so ./bin/appletviewer ./bin/idlj @@ -105,6 +109,17 @@ ACCEPTED_BIN_DIFF=" ./bin/xjc " +# Issue with __FILE__ usage in generated header files prevent clean fulldump diff of +# server jvm with old hotspot build. +KNOWN_FULLDUMP_DIFF=" +./lib/i386/server/libjvm.so +" +KNOWN_DIS_DIFF=" +./lib/i386/server/libjvm.so +" +DIS_DIFF_FILTER="$SED \ + -e 's/\(:\t\)\([0-9a-z]\{2,2\} \)\{1,7\}/\1/g' \ + -e 's/0x[0-9a-z]\{2,9\}//g'" fi if [ "$OPENJDK_TARGET_OS" = "linux" ] && [ "$OPENJDK_TARGET_CPU" = "x86_64" ]; then @@ -135,6 +150,7 @@ ACCEPTED_BIN_DIFF=" ./lib/amd64/libjsdt.so ./lib/amd64/libjsig.so ./lib/amd64/libmanagement.so +./lib/amd64/libnet.so ./lib/amd64/libnpt.so ./lib/amd64/libsaproc.so ./lib/amd64/libverify.so @@ -179,6 +195,12 @@ ACCEPTED_BIN_DIFF=" ./bin/xjc " +# Issue with __FILE__ usage in generated header files prevent clean fulldump diff of +# server jvm with old hotspot build. +KNOWN_FULLDUMP_DIFF=" +./lib/amd64/server/libjvm.so +" + fi if [ "$OPENJDK_TARGET_OS" = "solaris" ] && [ "$OPENJDK_TARGET_CPU" = "x86_64" ]; then @@ -299,14 +321,13 @@ SKIP_FULLDUMP_DIFF="true" # Filter random C++ symbol strings. # Some numbers differ randomly. -# Can't use space in these expressions as the shell will mess with them. DIS_DIFF_FILTER="$SED \ - -e 's/\.[a-zA-Z0-9_\$]\{15,15\}//g' \ - -e 's/\([0-9a-f][0-9a-f].\)\{2,8\}[0-9a-f][0-9a-f]//g' \ - -e 's/\(0x\)[0-9a-f]*\([,(>]\)/\1\2/g' \ - -e 's/\(0x\)[0-9a-f]*$/\1/g' \ - -e 's/\(\#.\)[0-9a-f]*\(.<\)/\1\2/g' \ - -e 's/[\.A-Za-z0-9%]\{16,16\}$//g'" + -e 's/\.[a-zA-Z0-9_\$]\{15\}//g' \ + -e 's/\(\# \)[0-9a-f]*\( <\)/\1\2/g' \ + -e 's/0x[0-9a-f]*$//g' \ + -e 's/0x[0-9a-f]*\([,(>]\)/\1/g' \ + -e 's/: [0-9a-f][0-9a-f]\( [0-9a-f][0-9a-f]\)\{2,10\}/: /g' \ + -e 's/ [\.A-Za-z0-9%@]\{16\}$/ /g'" fi @@ -425,18 +446,23 @@ ACCEPTED_SMALL_SIZE_DIFF=" ./bin/xjc " -# Filter random C++ symbol strings. # Some numbers differ randomly. DIS_DIFF_FILTER="$SED \ - -e 's/\$[a-zA-Z0-9_\$]\{15,15\}//g' \ - -e 's/[0-9a-f][0-9a-f].[0-9a-f][0-9a-f].[0-9a-f][0-9a-f].[0-9a-f][0-9a-f]//g' \ - -e 's/\(%g1,.0x\)[0-9a-f]*\(,.%g1\)/\1\2/g' \ - -e 's/\(!.\)[0-9a-f]*\(.\2/g' \ - -e 's/\!.[0-9a-f]\{1,4\} <_DYNAMIC+0x[0-9a-f]\{1,4\}>//g'" + -e 's/\$[a-zA-Z0-9_\$]\{15\}//g' \ + -e 's/: [0-9a-f][0-9a-f]\( [0-9a-f][0-9a-f]\)\{2,10\}/: /g' \ + -e 's/, [0-9a-fx\-]\{1,8\}/, /g' \ + -e 's/call [0-9a-f]\{7\}/call /g' \ + -e 's/0x[0-9a-f]\{1,8\}//g' \ + -e 's/\! [0-9a-f]\{1,8\} /! /g'" -# Some xor instructions end up with different args in the lib but not in the object files. -ACCEPTED_DIS_DIFF=" -./demo/jvmti/waiters/lib/libwaiters.so +# libjvm.so +# __FILE__ macro usage in debug.hpp causes differences between old and new +# hotspot builds in ad_sparc.o and ad_sparc_clone.o. The .o files compare +# equal when stripped, but at link time differences appear. Removing +# __FILE__ from ShouldNotCallThis() and ShouldNotReachHere() removes +# the differences. +KNOWN_DIS_DIFF=" +./lib/sparcv9/server/libjvm.so " SKIP_FULLDUMP_DIFF="true" @@ -634,11 +660,12 @@ ACCEPTED_BIN_DIFF=" SORT_SYMBOLS=" ./Contents/Home/lib/libsaproc.dylib ./lib/libsaproc.dylib +./lib/libjsig.dylib " ACCEPTED_SMALL_SIZE_DIFF="$ACCEPTED_BIN_DIFF" -DIS_DIFF_FILTER="$SED \ - -e 's/0x[0-9a-f]\{4,16\}//g'" +DIS_DIFF_FILTER="LANG=C $SED \ + -e 's/0x[0-9a-f]\{3,16\}//g' -e 's/^[0-9a-f]\{12,20\}//'" fi diff --git a/common/bin/jib.sh b/common/bin/jib.sh index 0fc60e9e76f..454d78ce1d7 100644 --- a/common/bin/jib.sh +++ b/common/bin/jib.sh @@ -32,7 +32,7 @@ installed_jib_script=${mydir}/../../.jib/jib install_data=${mydir}/../../.jib/.data setup_url() { - if [ -f "~/.config/jib/jib.conf" ]; then + if [ -f ~/.config/jib/jib.conf ]; then source ~/.config/jib/jib.conf fi @@ -50,6 +50,9 @@ setup_url() { if [ -n "${JIB_SERVER}" ]; then jib_server="${JIB_SERVER}" fi + if [ -n "${JIB_SERVER_MIRRORS}" ]; then + jib_server_mirrors="${JIB_SERVER_MIRRORS}" + fi if [ -n "${JIB_REPOSITORY}" ]; then jib_repository="${JIB_REPOSITORY}" fi @@ -70,8 +73,9 @@ setup_url() { jib_url="${JIB_URL}" data_string="${jib_url}" else - data_string="${jib_repository}/${jib_organization}/${jib_module}/${jib_revision}/${jib_module}-${jib_revision}.${jib_ext}" - jib_url="${jib_server}/${data_string}" + jib_path="${jib_repository}/${jib_organization}/${jib_module}/${jib_revision}/${jib_module}-${jib_revision}.${jib_ext}" + data_string="${jib_path}" + jib_url="${jib_server}/${jib_path}" fi } @@ -104,7 +108,25 @@ install_jib() { ${getcmd} ${jib_url} > "${installed_jib_script}.gz" if [ ! -s "${installed_jib_script}.gz" ]; then echo "Failed to download ${jib_url}" - exit 1 + if [ -n "${jib_path}" -a -n "${jib_server_mirrors}" ]; then + OLD_IFS="${IFS}" + IFS=" ," + for mirror in ${jib_server_mirrors}; do + echo "Trying mirror ${mirror}" + jib_url="${mirror}/${jib_path}" + ${getcmd} ${jib_url} > "${installed_jib_script}.gz" + if [ -s "${installed_jib_script}.gz" ]; then + echo "Download from mirror successful" + break + else + echo "Failed to download ${jib_url}" + fi + done + IFS="${OLD_IFS}" + fi + if [ ! -s "${installed_jib_script}.gz" ]; then + exit 1 + fi fi echo "Extracting JIB bootstrap script" rm -f "${installed_jib_script}" diff --git a/common/conf/jib-profiles.js b/common/conf/jib-profiles.js index b9c09fcd018..90d9621c6af 100644 --- a/common/conf/jib-profiles.js +++ b/common/conf/jib-profiles.js @@ -357,8 +357,8 @@ var getJibProfilesDependencies = function (input, common) { var devkit_platform_revisions = { linux_x64: "gcc4.9.2-OEL6.4+1.0", macosx_x64: "Xcode6.3-MacOSX10.9+1.0", - solaris_x64: "SS12u3-Solaris10u10+1.0", - solaris_sparcv9: "SS12u3-Solaris10u10+1.0", + solaris_x64: "SS12u4-Solaris11u1+1.0", + solaris_sparcv9: "SS12u4-Solaris11u1+1.0", windows_x64: "VS2013SP4+1.0" }; diff --git a/corba/.hgtags b/corba/.hgtags index 05a0a1e4423..9bd082d87fd 100644 --- a/corba/.hgtags +++ b/corba/.hgtags @@ -340,3 +340,5 @@ f7d70caad89ad0c43bb057bca0aad6f17ce05a6a jdk9-b92 fd038e8a16eec80d0d6b337d74a582790ed4b3ee jdk-9+95 feb1bd85d7990dcf5584ca9e53104269c01db006 jdk-9+96 10a482b863582376d4ca229090334b23b05159fc jdk-9+97 +ea285530245cf4e0edf0479121a41347d3030eba jdk-9+98 +180212ee1d8710691ba9944593dfc1ff3e4f1532 jdk-9+99 diff --git a/hotspot/.hgtags b/hotspot/.hgtags index 68035c56bdd..ac6cea4d970 100644 --- a/hotspot/.hgtags +++ b/hotspot/.hgtags @@ -500,3 +500,5 @@ a22b7c80529f5f05c847e932e017456e83c46233 jdk9-b94 0c79cf3cdf0904fc4a630b91b32904491e1ae430 jdk-9+95 a94bb7203596dd632486f1e3655fa5f70541dc08 jdk-9+96 de592ea5f7ba0f8a8c5afc03bd169f7690c72b6f jdk-9+97 +e5b1a23be1e105417ba1c4c576ab373eb3fa2c2b jdk-9+98 +f008e8cc10d5b3212fb22d58c96fa01d38654f19 jdk-9+99 diff --git a/hotspot/.mx.jvmci/mx_jvmci.py b/hotspot/.mx.jvmci/mx_jvmci.py index 0e3ac639bb3..d1069e6284c 100644 --- a/hotspot/.mx.jvmci/mx_jvmci.py +++ b/hotspot/.mx.jvmci/mx_jvmci.py @@ -677,12 +677,6 @@ class JVMCIArchiveParticipant: assert service self.services.setdefault(service, []).append(provider) return True - elif arcname.endswith('_OptionDescriptors.class'): - # Need to create service files for the providers of the - # jdk.vm.ci.options.Options service created by - # jdk.vm.ci.options.processor.OptionProcessor. - provider = arcname[:-len('.class'):].replace('/', '.') - self.services.setdefault('jdk.vm.ci.options.OptionDescriptors', []).append(provider) return False def __addsrc__(self, arcname, contents): @@ -761,21 +755,6 @@ class JVMCI9JDKConfig(mx.JDKConfig): if jacocoArgs: args = jacocoArgs + args - # Support for -G: options - def translateGOption(arg): - if arg.startswith('-G:+'): - if '=' in arg: - mx.abort('Mixing + and = in -G: option specification: ' + arg) - arg = '-Djvmci.option.' + arg[len('-G:+'):] + '=true' - elif arg.startswith('-G:-'): - if '=' in arg: - mx.abort('Mixing - and = in -G: option specification: ' + arg) - arg = '-Djvmci.option.' + arg[len('-G:+'):] + '=false' - elif arg.startswith('-G:'): - arg = '-Djvmci.option.' + arg[len('-G:'):] - return arg - args = map(translateGOption, args) - args = ['-Xbootclasspath/p:' + dep.classpath_repr() for dep in _jvmci_bootclasspath_prepends] + args jvmciModeArgs = _jvmciModes[_vm.jvmciMode] diff --git a/hotspot/.mx.jvmci/suite.py b/hotspot/.mx.jvmci/suite.py index 3fd3a3cddc1..05d0027a8bb 100644 --- a/hotspot/.mx.jvmci/suite.py +++ b/hotspot/.mx.jvmci/suite.py @@ -109,7 +109,6 @@ suite = { "jdk.vm.ci.code", ], "checkstyle" : "jdk.vm.ci.service", - "annotationProcessors" : ["JVMCI_OPTIONS_PROCESSOR"], "javaCompliance" : "1.8", "workingSets" : "API,JVMCI", }, @@ -135,40 +134,17 @@ suite = { "workingSets" : "JVMCI", }, - "jdk.vm.ci.options" : { - "subDir" : "src/jdk.vm.ci/share/classes", - "sourceDirs" : ["src"], - "checkstyle" : "jdk.vm.ci.service", - "dependencies" : ["jdk.vm.ci.inittimer"], - "javaCompliance" : "1.8", - "workingSets" : "JVMCI", - }, - - "jdk.vm.ci.options.processor" : { - "subDir" : "src/jdk.vm.ci/share/classes", - "sourceDirs" : ["src"], - "dependencies" : [ - "jdk.vm.ci.options", - ], - "checkstyle" : "jdk.vm.ci.service", - "javaCompliance" : "1.8", - "workingSets" : "JVMCI,Codegen", - }, - - "jdk.vm.ci.options.test" : { - "subDir" : "test/compiler/jvmci", - "sourceDirs" : ["src"], - "dependencies" : [ - "jdk.vm.ci.options", - "mx:JUNIT", - ], - "checkstyle" : "jdk.vm.ci.service", - "javaCompliance" : "1.8", - "workingSets" : "JVMCI", - }, - # ------------- JVMCI:HotSpot ------------- + "jdk.vm.ci.aarch64" : { + "subDir" : "src/jdk.vm.ci/share/classes", + "sourceDirs" : ["src"], + "dependencies" : ["jdk.vm.ci.code"], + "checkstyle" : "jdk.vm.ci.service", + "javaCompliance" : "1.8", + "workingSets" : "JVMCI,AArch64", + }, + "jdk.vm.ci.amd64" : { "subDir" : "src/jdk.vm.ci/share/classes", "sourceDirs" : ["src"], @@ -191,15 +167,12 @@ suite = { "subDir" : "src/jdk.vm.ci/share/classes", "sourceDirs" : ["src"], "dependencies" : [ - "jdk.vm.ci.options", "jdk.vm.ci.hotspotvmconfig", "jdk.vm.ci.common", + "jdk.vm.ci.inittimer", "jdk.vm.ci.runtime", "jdk.vm.ci.service", ], - "annotationProcessors" : [ - "JVMCI_OPTIONS_PROCESSOR", - ], "checkstyle" : "jdk.vm.ci.service", "javaCompliance" : "1.8", "workingSets" : "JVMCI", @@ -213,6 +186,21 @@ suite = { "workingSets" : "JVMCI,HotSpot", }, + "jdk.vm.ci.hotspot.aarch64" : { + "subDir" : "src/jdk.vm.ci/share/classes", + "sourceDirs" : ["src"], + "dependencies" : [ + "jdk.vm.ci.aarch64", + "jdk.vm.ci.hotspot", + ], + "checkstyle" : "jdk.vm.ci.service", + "annotationProcessors" : [ + "JVMCI_SERVICE_PROCESSOR", + ], + "javaCompliance" : "1.8", + "workingSets" : "JVMCI,HotSpot,AArch64", + }, + "jdk.vm.ci.hotspot.amd64" : { "subDir" : "src/jdk.vm.ci/share/classes", "sourceDirs" : ["src"], @@ -258,22 +246,17 @@ suite = { "dependencies" : ["jdk.vm.ci.service"], }, - "JVMCI_OPTIONS" : { - "subDir" : "src/jdk.vm.ci/share/classes", - "dependencies" : ["jdk.vm.ci.options"], - }, - "JVMCI_API" : { "subDir" : "src/jdk.vm.ci/share/classes", "dependencies" : [ "jdk.vm.ci.inittimer", "jdk.vm.ci.runtime", "jdk.vm.ci.common", + "jdk.vm.ci.aarch64", "jdk.vm.ci.amd64", "jdk.vm.ci.sparc", ], "distDependencies" : [ - "JVMCI_OPTIONS", "JVMCI_SERVICE", ], }, @@ -288,6 +271,7 @@ suite = { "JVMCI_HOTSPOT" : { "subDir" : "src/jdk.vm.ci/share/classes", "dependencies" : [ + "jdk.vm.ci.hotspot.aarch64", "jdk.vm.ci.hotspot.amd64", "jdk.vm.ci.hotspot.sparc", ], @@ -301,7 +285,6 @@ suite = { "JVMCI_TEST" : { "subDir" : "test/compiler/jvmci", "dependencies" : [ - "jdk.vm.ci.options.test", "jdk.vm.ci.runtime.test", ], "distDependencies" : [ @@ -310,13 +293,6 @@ suite = { "exclude" : ["mx:JUNIT"], }, - "JVMCI_OPTIONS_PROCESSOR" : { - "subDir" : "src/jdk.vm.ci/share/classes", - "dependencies" : ["jdk.vm.ci.options.processor"], - "distDependencies" : [ - "JVMCI_OPTIONS", - ], - }, "JVMCI_SERVICE_PROCESSOR" : { "subDir" : "src/jdk.vm.ci/share/classes", @@ -332,25 +308,23 @@ suite = { "subDir" : "src/jdk.vm.ci/share/classes", "overlaps" : [ "JVMCI_API", - "JVMCI_OPTIONS", "JVMCI_SERVICE", "JVMCI_HOTSPOT", "JVMCI_HOTSPOTVMCONFIG", "JVMCI_SERVICE_PROCESSOR", - "JVMCI_OPTIONS_PROCESSOR" ], "dependencies" : [ - "jdk.vm.ci.options", "jdk.vm.ci.service", "jdk.vm.ci.inittimer", "jdk.vm.ci.runtime", "jdk.vm.ci.common", + "jdk.vm.ci.aarch64", "jdk.vm.ci.amd64", "jdk.vm.ci.sparc", "jdk.vm.ci.hotspotvmconfig", + "jdk.vm.ci.hotspot.aarch64", "jdk.vm.ci.hotspot.amd64", "jdk.vm.ci.hotspot.sparc", - "jdk.vm.ci.options.processor", "jdk.vm.ci.service.processor" ], }, diff --git a/hotspot/make/aix/Makefile b/hotspot/make/aix/Makefile index 4e0db05251c..77e54d08dc0 100644 --- a/hotspot/make/aix/Makefile +++ b/hotspot/make/aix/Makefile @@ -1,6 +1,6 @@ # # Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. -# Copyright 2012, 2013 SAP AG. All rights reserved. +# Copyright 2012, 2015 SAP AG. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -61,10 +61,6 @@ ifndef CC_INTERP FORCE_TIERED=1 endif endif -# C1 is not ported on ppc64(le), so we cannot build a tiered VM: -ifneq (,$(filter $(ARCH),ppc64 pp64le)) - FORCE_TIERED=0 -endif ifdef LP64 ifeq ("$(filter $(LP64_ARCH),$(BUILDARCH))","") diff --git a/hotspot/make/aix/makefiles/fastdebug.make b/hotspot/make/aix/makefiles/fastdebug.make index 33bf50940b9..d7feb025617 100644 --- a/hotspot/make/aix/makefiles/fastdebug.make +++ b/hotspot/make/aix/makefiles/fastdebug.make @@ -68,5 +68,5 @@ MAPFILE = $(GAMMADIR)/make/aix/makefiles/mapfile-vers-debug LFLAGS_QIPA= VERSION = optimized -SYSDEFS += -DASSERT -DFASTDEBUG +SYSDEFS += -DASSERT PICFLAGS = DEFAULT diff --git a/hotspot/make/aix/makefiles/tiered.make b/hotspot/make/aix/makefiles/tiered.make new file mode 100644 index 00000000000..992d59d5241 --- /dev/null +++ b/hotspot/make/aix/makefiles/tiered.make @@ -0,0 +1,32 @@ +# +# Copyright (c) 2006, 2015, Oracle and/or its affiliates. All rights reserved. +# Copyright 2012, 2015 SAP AG. All rights reserved. +# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. +# +# This code is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License version 2 only, as +# published by the Free Software Foundation. +# +# This code is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# version 2 for more details (a copy is included in the LICENSE file that +# accompanied this code). +# +# You should have received a copy of the GNU General Public License version +# 2 along with this work; if not, write to the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +# +# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA +# or visit www.oracle.com if you need additional information or have any +# questions. +# +# + +# Sets make macros for making tiered version of VM + +TYPE=TIERED + +VM_SUBDIR = server + +CFLAGS += -DCOMPILER2 -DCOMPILER1 diff --git a/hotspot/make/excludeSrc.make b/hotspot/make/excludeSrc.make index 9feb96861ee..cab00edf842 100644 --- a/hotspot/make/excludeSrc.make +++ b/hotspot/make/excludeSrc.make @@ -107,8 +107,8 @@ ifeq ($(INCLUDE_NMT), false) memTracker.cpp nmtDCmd.cpp mallocSiteTable.cpp endif -ifneq (,$(findstring $(Platform_arch_model), x86_64, sparc)) - # JVMCI is supported only on x86_64 and SPARC. +ifneq (,$(findstring $(Platform_arch_model), aarch64, arm_64, sparc, x86_64)) + # JVMCI is supported else INCLUDE_JVMCI := false endif diff --git a/hotspot/make/gensrc/Gensrc-jdk.vm.ci.gmk b/hotspot/make/gensrc/Gensrc-jdk.vm.ci.gmk index 654a7f8fd32..969330eec19 100644 --- a/hotspot/make/gensrc/Gensrc-jdk.vm.ci.gmk +++ b/hotspot/make/gensrc/Gensrc-jdk.vm.ci.gmk @@ -36,15 +36,6 @@ SRC_DIR := $(HOTSPOT_TOPDIR)/src/jdk.vm.ci/share/classes ################################################################################ # Compile the annotation processor -$(eval $(call SetupJavaCompilation, BUILD_JVMCI_OPTIONS, \ - SETUP := GENERATE_OLDBYTECODE, \ - SRC := $(SRC_DIR)/jdk.vm.ci.options/src \ - $(SRC_DIR)/jdk.vm.ci.options.processor/src \ - $(SRC_DIR)/jdk.vm.ci.inittimer/src, \ - BIN := $(BUILDTOOLS_OUTPUTDIR)/jvmci_options, \ - JAR := $(BUILDTOOLS_OUTPUTDIR)/jdk.vm.ci.options.jar, \ -)) - $(eval $(call SetupJavaCompilation, BUILD_JVMCI_SERVICE, \ SETUP := GENERATE_OLDBYTECODE, \ SRC := $(SRC_DIR)/jdk.vm.ci.service/src \ @@ -57,6 +48,7 @@ $(eval $(call SetupJavaCompilation, BUILD_JVMCI_SERVICE, \ PROC_SRC_SUBDIRS := \ jdk.vm.ci.hotspot \ + jdk.vm.ci.hotspot.aarch64 \ jdk.vm.ci.hotspot.amd64 \ jdk.vm.ci.hotspot.sparc \ jdk.vm.ci.runtime \ @@ -69,11 +61,10 @@ PROC_SRCS := $(filter %.java, $(call CacheFind, $(PROC_SRC_DIRS))) ALL_SRC_DIRS := $(wildcard $(SRC_DIR)/*/src) SOURCEPATH := $(call PathList, $(ALL_SRC_DIRS)) PROCESSOR_PATH := $(call PathList, \ - $(BUILDTOOLS_OUTPUTDIR)/jdk.vm.ci.options.jar \ $(BUILDTOOLS_OUTPUTDIR)/jdk.vm.ci.service.jar) $(GENSRC_DIR)/_gensrc_proc_done: $(PROC_SRCS) \ - $(BUILD_JVMCI_OPTIONS) $(BUILD_JVMCI_SERVICE) + $(BUILD_JVMCI_SERVICE) $(MKDIR) -p $(@D) $(eval $(call ListPathsSafely,PROC_SRCS,$(@D)/_gensrc_proc_files)) $(JAVA_SMALL) $(NEW_JAVAC) \ @@ -91,15 +82,6 @@ TARGETS += $(GENSRC_DIR)/_gensrc_proc_done ################################################################################ -$(GENSRC_DIR)/META-INF/services/jdk.vm.ci.options.OptionDescriptors: \ - $(GENSRC_DIR)/_gensrc_proc_done - $(MKDIR) -p $(@D) - $(FIND) $(GENSRC_DIR) -name '*_OptionDescriptors.java' | $(SED) 's:.*/jdk\.vm\.ci/\(.*\)\.java:\1:' | $(TR) '/' '.' > $@ - -TARGETS += $(GENSRC_DIR)/META-INF/services/jdk.vm.ci.options.OptionDescriptors - -################################################################################ - $(GENSRC_DIR)/_providers_converted: $(GENSRC_DIR)/_gensrc_proc_done $(MKDIR) -p $(GENSRC_DIR)/META-INF/services ($(CD) $(GENSRC_DIR)/META-INF/jvmci.providers && \ diff --git a/hotspot/make/linux/Makefile b/hotspot/make/linux/Makefile index 52aa0305f7a..dba28625eec 100644 --- a/hotspot/make/linux/Makefile +++ b/hotspot/make/linux/Makefile @@ -57,14 +57,6 @@ ifndef CC_INTERP FORCE_TIERED=1 endif endif -# C1 is not ported on ppc64, so we cannot build a tiered VM: -# Notice: after 8046471 ARCH will be 'ppc' for top-level ppc64 builds but -# 'ppc64' for HotSpot-only ppc64 builds. Need to detect both variants here! -ifneq (,$(findstring $(ARCH), ppc ppc64)) - ifeq ($(ARCH_DATA_MODEL), 64) - FORCE_TIERED=0 - endif -endif ifdef LP64 ifeq ("$(filter $(LP64_ARCH),$(BUILDARCH))","") diff --git a/hotspot/make/test/JtregNative.gmk b/hotspot/make/test/JtregNative.gmk index 09c48d77aba..109c04d37fb 100644 --- a/hotspot/make/test/JtregNative.gmk +++ b/hotspot/make/test/JtregNative.gmk @@ -46,6 +46,8 @@ BUILD_HOTSPOT_JTREG_NATIVE_SRC := \ $(HOTSPOT_TOPDIR)/test/runtime/jni/8033445 \ $(HOTSPOT_TOPDIR)/test/runtime/jni/ToStringInInterfaceTest \ $(HOTSPOT_TOPDIR)/test/runtime/SameObject \ + $(HOTSPOT_TOPDIR)/test/compiler/floatingpoint/ \ + $(HOTSPOT_TOPDIR)/test/compiler/calls \ # # Add conditional directories here when needed. diff --git a/hotspot/src/cpu/aarch64/vm/aarch64.ad b/hotspot/src/cpu/aarch64/vm/aarch64.ad index afa30b74491..f19eb0e71c7 100644 --- a/hotspot/src/cpu/aarch64/vm/aarch64.ad +++ b/hotspot/src/cpu/aarch64/vm/aarch64.ad @@ -3484,10 +3484,14 @@ int Matcher::regnum_to_fpu_offset(int regnum) return 0; } -bool Matcher::is_short_branch_offset(int rule, int br_size, int offset) -{ - Unimplemented(); - return false; +// Is this branch offset short enough that a short branch can be used? +// +// NOTE: If the platform does not provide any short branch variants, then +// this method should return false for offset 0. +bool Matcher::is_short_branch_offset(int rule, int br_size, int offset) { + // The passed offset is relative to address of the branch. + + return (-32768 <= offset && offset < 32768); } const bool Matcher::isSimpleConstant64(jlong value) { @@ -4667,17 +4671,12 @@ encode %{ if (!_method) { // A call to a runtime wrapper, e.g. new, new_typeArray_Java, uncommon_trap. call = __ trampoline_call(Address(addr, relocInfo::runtime_call_type), &cbuf); - } else if (_optimized_virtual) { - call = __ trampoline_call(Address(addr, relocInfo::opt_virtual_call_type), &cbuf); } else { - call = __ trampoline_call(Address(addr, relocInfo::static_call_type), &cbuf); - } - if (call == NULL) { - ciEnv::current()->record_failure("CodeCache is full"); - return; - } + int method_index = resolved_method_index(cbuf); + RelocationHolder rspec = _optimized_virtual ? opt_virtual_call_Relocation::spec(method_index) + : static_call_Relocation::spec(method_index); + call = __ trampoline_call(Address(addr, rspec), &cbuf); - if (_method) { // Emit stub for static call address stub = CompiledStaticCall::emit_to_interp_stub(cbuf); if (stub == NULL) { @@ -4685,11 +4684,16 @@ encode %{ return; } } + if (call == NULL) { + ciEnv::current()->record_failure("CodeCache is full"); + return; + } %} enc_class aarch64_enc_java_dynamic_call(method meth) %{ MacroAssembler _masm(&cbuf); - address call = __ ic_call((address)$meth$$method); + int method_index = resolved_method_index(cbuf); + address call = __ ic_call((address)$meth$$method, method_index); if (call == NULL) { ciEnv::current()->record_failure("CodeCache is full"); return; @@ -13845,7 +13849,8 @@ instruct cmpP_narrowOop_imm0_branch(cmpOp cmp, iRegN oop, immP0 zero, label labl // Test bit and Branch -instruct cmpL_branch_sign(cmpOp cmp, iRegL op1, immL0 op2, label labl, rFlagsReg cr) %{ +// Patterns for short (< 32KiB) variants +instruct cmpL_branch_sign(cmpOp cmp, iRegL op1, immL0 op2, label labl) %{ match(If cmp (CmpL op1 op2)); predicate(n->in(1)->as_Bool()->_test._test == BoolTest::lt || n->in(1)->as_Bool()->_test._test == BoolTest::ge); @@ -13855,16 +13860,15 @@ instruct cmpL_branch_sign(cmpOp cmp, iRegL op1, immL0 op2, label labl, rFlagsReg format %{ "cb$cmp $op1, $labl # long" %} ins_encode %{ Label* L = $labl$$label; - Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; - if (cond == Assembler::LT) - __ tbnz($op1$$Register, 63, *L); - else - __ tbz($op1$$Register, 63, *L); + Assembler::Condition cond = + ((Assembler::Condition)$cmp$$cmpcode == Assembler::LT) ? Assembler::NE : Assembler::EQ; + __ tbr(cond, $op1$$Register, 63, *L); %} ins_pipe(pipe_cmp_branch); + ins_short_branch(1); %} -instruct cmpI_branch_sign(cmpOp cmp, iRegIorL2I op1, immI0 op2, label labl, rFlagsReg cr) %{ +instruct cmpI_branch_sign(cmpOp cmp, iRegIorL2I op1, immI0 op2, label labl) %{ match(If cmp (CmpI op1 op2)); predicate(n->in(1)->as_Bool()->_test._test == BoolTest::lt || n->in(1)->as_Bool()->_test._test == BoolTest::ge); @@ -13874,16 +13878,15 @@ instruct cmpI_branch_sign(cmpOp cmp, iRegIorL2I op1, immI0 op2, label labl, rFla format %{ "cb$cmp $op1, $labl # int" %} ins_encode %{ Label* L = $labl$$label; - Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; - if (cond == Assembler::LT) - __ tbnz($op1$$Register, 31, *L); - else - __ tbz($op1$$Register, 31, *L); + Assembler::Condition cond = + ((Assembler::Condition)$cmp$$cmpcode == Assembler::LT) ? Assembler::NE : Assembler::EQ; + __ tbr(cond, $op1$$Register, 31, *L); %} ins_pipe(pipe_cmp_branch); + ins_short_branch(1); %} -instruct cmpL_branch_bit(cmpOp cmp, iRegL op1, immL op2, immL0 op3, label labl, rFlagsReg cr) %{ +instruct cmpL_branch_bit(cmpOp cmp, iRegL op1, immL op2, immL0 op3, label labl) %{ match(If cmp (CmpL (AndL op1 op2) op3)); predicate((n->in(1)->as_Bool()->_test._test == BoolTest::ne || n->in(1)->as_Bool()->_test._test == BoolTest::eq) @@ -13896,15 +13899,13 @@ instruct cmpL_branch_bit(cmpOp cmp, iRegL op1, immL op2, immL0 op3, label labl, Label* L = $labl$$label; Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; int bit = exact_log2($op2$$constant); - if (cond == Assembler::EQ) - __ tbz($op1$$Register, bit, *L); - else - __ tbnz($op1$$Register, bit, *L); + __ tbr(cond, $op1$$Register, bit, *L); %} ins_pipe(pipe_cmp_branch); + ins_short_branch(1); %} -instruct cmpI_branch_bit(cmpOp cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl, rFlagsReg cr) %{ +instruct cmpI_branch_bit(cmpOp cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{ match(If cmp (CmpI (AndI op1 op2) op3)); predicate((n->in(1)->as_Bool()->_test._test == BoolTest::ne || n->in(1)->as_Bool()->_test._test == BoolTest::eq) @@ -13917,10 +13918,79 @@ instruct cmpI_branch_bit(cmpOp cmp, iRegIorL2I op1, immI op2, immI0 op3, label l Label* L = $labl$$label; Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; int bit = exact_log2($op2$$constant); - if (cond == Assembler::EQ) - __ tbz($op1$$Register, bit, *L); - else - __ tbnz($op1$$Register, bit, *L); + __ tbr(cond, $op1$$Register, bit, *L); + %} + ins_pipe(pipe_cmp_branch); + ins_short_branch(1); +%} + +// And far variants +instruct far_cmpL_branch_sign(cmpOp cmp, iRegL op1, immL0 op2, label labl) %{ + match(If cmp (CmpL op1 op2)); + predicate(n->in(1)->as_Bool()->_test._test == BoolTest::lt + || n->in(1)->as_Bool()->_test._test == BoolTest::ge); + effect(USE labl); + + ins_cost(BRANCH_COST); + format %{ "cb$cmp $op1, $labl # long" %} + ins_encode %{ + Label* L = $labl$$label; + Assembler::Condition cond = + ((Assembler::Condition)$cmp$$cmpcode == Assembler::LT) ? Assembler::NE : Assembler::EQ; + __ tbr(cond, $op1$$Register, 63, *L, /*far*/true); + %} + ins_pipe(pipe_cmp_branch); +%} + +instruct far_cmpI_branch_sign(cmpOp cmp, iRegIorL2I op1, immI0 op2, label labl) %{ + match(If cmp (CmpI op1 op2)); + predicate(n->in(1)->as_Bool()->_test._test == BoolTest::lt + || n->in(1)->as_Bool()->_test._test == BoolTest::ge); + effect(USE labl); + + ins_cost(BRANCH_COST); + format %{ "cb$cmp $op1, $labl # int" %} + ins_encode %{ + Label* L = $labl$$label; + Assembler::Condition cond = + ((Assembler::Condition)$cmp$$cmpcode == Assembler::LT) ? Assembler::NE : Assembler::EQ; + __ tbr(cond, $op1$$Register, 31, *L, /*far*/true); + %} + ins_pipe(pipe_cmp_branch); +%} + +instruct far_cmpL_branch_bit(cmpOp cmp, iRegL op1, immL op2, immL0 op3, label labl) %{ + match(If cmp (CmpL (AndL op1 op2) op3)); + predicate((n->in(1)->as_Bool()->_test._test == BoolTest::ne + || n->in(1)->as_Bool()->_test._test == BoolTest::eq) + && is_power_of_2(n->in(2)->in(1)->in(2)->get_long())); + effect(USE labl); + + ins_cost(BRANCH_COST); + format %{ "tb$cmp $op1, $op2, $labl" %} + ins_encode %{ + Label* L = $labl$$label; + Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; + int bit = exact_log2($op2$$constant); + __ tbr(cond, $op1$$Register, bit, *L, /*far*/true); + %} + ins_pipe(pipe_cmp_branch); +%} + +instruct far_cmpI_branch_bit(cmpOp cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{ + match(If cmp (CmpI (AndI op1 op2) op3)); + predicate((n->in(1)->as_Bool()->_test._test == BoolTest::ne + || n->in(1)->as_Bool()->_test._test == BoolTest::eq) + && is_power_of_2(n->in(2)->in(1)->in(2)->get_int())); + effect(USE labl); + + ins_cost(BRANCH_COST); + format %{ "tb$cmp $op1, $op2, $labl" %} + ins_encode %{ + Label* L = $labl$$label; + Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode; + int bit = exact_log2($op2$$constant); + __ tbr(cond, $op1$$Register, bit, *L, /*far*/true); %} ins_pipe(pipe_cmp_branch); %} @@ -15318,6 +15388,124 @@ instruct vmul2D(vecX dst, vecX src1, vecX src2) ins_pipe(pipe_class_default); %} +// --------------------------------- MLA -------------------------------------- + +instruct vmla4S(vecD dst, vecD src1, vecD src2) +%{ + predicate(n->as_Vector()->length() == 2 || + n->as_Vector()->length() == 4); + match(Set dst (AddVS dst (MulVS src1 src2))); + ins_cost(INSN_COST); + format %{ "mlav $dst,$src1,$src2\t# vector (4H)" %} + ins_encode %{ + __ mlav(as_FloatRegister($dst$$reg), __ T4H, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmla8S(vecX dst, vecX src1, vecX src2) +%{ + predicate(n->as_Vector()->length() == 8); + match(Set dst (AddVS dst (MulVS src1 src2))); + ins_cost(INSN_COST); + format %{ "mlav $dst,$src1,$src2\t# vector (8H)" %} + ins_encode %{ + __ mlav(as_FloatRegister($dst$$reg), __ T8H, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmla2I(vecD dst, vecD src1, vecD src2) +%{ + predicate(n->as_Vector()->length() == 2); + match(Set dst (AddVI dst (MulVI src1 src2))); + ins_cost(INSN_COST); + format %{ "mlav $dst,$src1,$src2\t# vector (2S)" %} + ins_encode %{ + __ mlav(as_FloatRegister($dst$$reg), __ T2S, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmla4I(vecX dst, vecX src1, vecX src2) +%{ + predicate(n->as_Vector()->length() == 4); + match(Set dst (AddVI dst (MulVI src1 src2))); + ins_cost(INSN_COST); + format %{ "mlav $dst,$src1,$src2\t# vector (4S)" %} + ins_encode %{ + __ mlav(as_FloatRegister($dst$$reg), __ T4S, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +// --------------------------------- MLS -------------------------------------- + +instruct vmls4S(vecD dst, vecD src1, vecD src2) +%{ + predicate(n->as_Vector()->length() == 2 || + n->as_Vector()->length() == 4); + match(Set dst (SubVS dst (MulVS src1 src2))); + ins_cost(INSN_COST); + format %{ "mlsv $dst,$src1,$src2\t# vector (4H)" %} + ins_encode %{ + __ mlsv(as_FloatRegister($dst$$reg), __ T4H, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmls8S(vecX dst, vecX src1, vecX src2) +%{ + predicate(n->as_Vector()->length() == 8); + match(Set dst (SubVS dst (MulVS src1 src2))); + ins_cost(INSN_COST); + format %{ "mlsv $dst,$src1,$src2\t# vector (8H)" %} + ins_encode %{ + __ mlsv(as_FloatRegister($dst$$reg), __ T8H, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmls2I(vecD dst, vecD src1, vecD src2) +%{ + predicate(n->as_Vector()->length() == 2); + match(Set dst (SubVI dst (MulVI src1 src2))); + ins_cost(INSN_COST); + format %{ "mlsv $dst,$src1,$src2\t# vector (2S)" %} + ins_encode %{ + __ mlsv(as_FloatRegister($dst$$reg), __ T2S, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + +instruct vmls4I(vecX dst, vecX src1, vecX src2) +%{ + predicate(n->as_Vector()->length() == 4); + match(Set dst (SubVI dst (MulVI src1 src2))); + ins_cost(INSN_COST); + format %{ "mlsv $dst,$src1,$src2\t# vector (4S)" %} + ins_encode %{ + __ mlsv(as_FloatRegister($dst$$reg), __ T4S, + as_FloatRegister($src1$$reg), + as_FloatRegister($src2$$reg)); + %} + ins_pipe(pipe_class_default); +%} + // --------------------------------- DIV -------------------------------------- instruct vdiv2F(vecD dst, vecD src1, vecD src2) diff --git a/hotspot/src/cpu/aarch64/vm/assembler_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/assembler_aarch64.hpp index dcd99c341ce..ca617716562 100644 --- a/hotspot/src/cpu/aarch64/vm/assembler_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/assembler_aarch64.hpp @@ -135,15 +135,10 @@ REGISTER_DECLARATION(Register, rlocals, r24); // bytecode pointer REGISTER_DECLARATION(Register, rbcp, r22); // Dispatch table base -REGISTER_DECLARATION(Register, rdispatch, r21); +REGISTER_DECLARATION(Register, rdispatch, r21); // Java stack pointer REGISTER_DECLARATION(Register, esp, r20); -// TODO : x86 uses rbp to save SP in method handle code -// we may need to do the same with fp -// JSR 292 fixed register usages: -//REGISTER_DECLARATION(Register, r_mh_SP_save, r29); - #define assert_cond(ARG1) assert(ARG1, #ARG1) namespace asm_util { @@ -551,6 +546,7 @@ class Address VALUE_OBJ_CLASS_SPEC { size = 0; break; default: ShouldNotReachHere(); + size = 0; // unreachable } } else { size = i->get(31, 31); @@ -2041,6 +2037,8 @@ public: INSN(addv, 0, 0b100001); INSN(subv, 1, 0b100001); INSN(mulv, 0, 0b100111); + INSN(mlav, 0, 0b100101); + INSN(mlsv, 1, 0b100101); INSN(sshl, 0, 0b010001); INSN(ushl, 1, 0b010001); diff --git a/hotspot/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp index 119d113aac5..210b29e1e75 100644 --- a/hotspot/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp @@ -173,6 +173,7 @@ static jlong as_long(LIR_Opr data) { break; default: ShouldNotReachHere(); + result = 0; // unreachable } return result; } @@ -720,6 +721,7 @@ void LIR_Assembler::const2mem(LIR_Opr src, LIR_Opr dest, BasicType type, CodeEmi break; default: ShouldNotReachHere(); + insn = &Assembler::str; // unreachable } if (info) add_debug_info_for_null_check_here(info); @@ -1110,6 +1112,7 @@ void LIR_Assembler::emit_opBranch(LIR_OpBranch* op) { case lir_cond_greaterEqual: acond = (is_unordered ? Assembler::HS : Assembler::GE); break; case lir_cond_greater: acond = (is_unordered ? Assembler::HI : Assembler::GT); break; default: ShouldNotReachHere(); + acond = Assembler::EQ; // unreachable } } else { switch (op->cond()) { @@ -1121,7 +1124,8 @@ void LIR_Assembler::emit_opBranch(LIR_OpBranch* op) { case lir_cond_greater: acond = Assembler::GT; break; case lir_cond_belowEqual: acond = Assembler::LS; break; case lir_cond_aboveEqual: acond = Assembler::HS; break; - default: ShouldNotReachHere(); + default: ShouldNotReachHere(); + acond = Assembler::EQ; // unreachable } } __ br(acond,*(op->label())); @@ -1313,7 +1317,9 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L ciMethodData* md; ciProfileData* data; - if (op->should_profile()) { + const bool should_profile = op->should_profile(); + + if (should_profile) { ciMethod* method = op->profiled_method(); assert(method != NULL, "Should have method"); int bci = op->profiled_bci(); @@ -1324,8 +1330,8 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L assert(data->is_ReceiverTypeData(), "need ReceiverTypeData for type check"); } Label profile_cast_success, profile_cast_failure; - Label *success_target = op->should_profile() ? &profile_cast_success : success; - Label *failure_target = op->should_profile() ? &profile_cast_failure : failure; + Label *success_target = should_profile ? &profile_cast_success : success; + Label *failure_target = should_profile ? &profile_cast_failure : failure; if (obj == k_RInfo) { k_RInfo = dst; @@ -1341,7 +1347,7 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L assert_different_registers(obj, k_RInfo, klass_RInfo); - if (op->should_profile()) { + if (should_profile) { Label not_null; __ cbnz(obj, not_null); // Object is null; update MDO and exit @@ -1413,7 +1419,7 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L // successful cast, fall through to profile or jump } } - if (op->should_profile()) { + if (should_profile) { Register mdo = klass_RInfo, recv = k_RInfo; __ bind(profile_cast_success); __ mov_metadata(mdo, md->constant_encoding()); @@ -1438,6 +1444,8 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) { + const bool should_profile = op->should_profile(); + LIR_Code code = op->code(); if (code == lir_store_check) { Register value = op->object()->as_register(); @@ -1452,7 +1460,7 @@ void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) { ciMethodData* md; ciProfileData* data; - if (op->should_profile()) { + if (should_profile) { ciMethod* method = op->profiled_method(); assert(method != NULL, "Should have method"); int bci = op->profiled_bci(); @@ -1463,10 +1471,10 @@ void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) { assert(data->is_ReceiverTypeData(), "need ReceiverTypeData for type check"); } Label profile_cast_success, profile_cast_failure, done; - Label *success_target = op->should_profile() ? &profile_cast_success : &done; - Label *failure_target = op->should_profile() ? &profile_cast_failure : stub->entry(); + Label *success_target = should_profile ? &profile_cast_success : &done; + Label *failure_target = should_profile ? &profile_cast_failure : stub->entry(); - if (op->should_profile()) { + if (should_profile) { Label not_null; __ cbnz(value, not_null); // Object is null; update MDO and exit @@ -1502,7 +1510,7 @@ void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) { __ cbzw(k_RInfo, *failure_target); // fall through to the success case - if (op->should_profile()) { + if (should_profile) { Register mdo = klass_RInfo, recv = k_RInfo; __ bind(profile_cast_success); __ mov_metadata(mdo, md->constant_encoding()); @@ -1621,9 +1629,10 @@ void LIR_Assembler::cmove(LIR_Condition condition, LIR_Opr opr1, LIR_Opr opr2, L case lir_cond_lessEqual: acond = Assembler::LE; ncond = Assembler::GT; break; case lir_cond_greaterEqual: acond = Assembler::GE; ncond = Assembler::LT; break; case lir_cond_greater: acond = Assembler::GT; ncond = Assembler::LE; break; - case lir_cond_belowEqual: Unimplemented(); break; - case lir_cond_aboveEqual: Unimplemented(); break; + case lir_cond_belowEqual: + case lir_cond_aboveEqual: default: ShouldNotReachHere(); + acond = Assembler::EQ; ncond = Assembler::NE; // unreachable } assert(result->is_single_cpu() || result->is_double_cpu(), @@ -1724,6 +1733,7 @@ void LIR_Assembler::arith_op(LIR_Code code, LIR_Opr left, LIR_Opr right, LIR_Opr break; default: ShouldNotReachHere(); + c = 0; // unreachable break; } @@ -1926,6 +1936,7 @@ void LIR_Assembler::comp_op(LIR_Condition condition, LIR_Opr opr1, LIR_Opr opr2, break; default: ShouldNotReachHere(); + imm = 0; // unreachable break; } @@ -3123,6 +3134,9 @@ void LIR_Assembler::atomic_op(LIR_Code code, LIR_Opr src, LIR_Opr data, LIR_Opr break; default: ShouldNotReachHere(); + lda = &MacroAssembler::ldaxr; + add = &MacroAssembler::add; + stl = &MacroAssembler::stlxr; // unreachable } switch (code) { diff --git a/hotspot/src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp index 21c7cfc93ff..191c4e45571 100644 --- a/hotspot/src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp @@ -238,6 +238,7 @@ LIR_Opr LIRGenerator::load_immediate(int x, BasicType type) { } } else { ShouldNotReachHere(); + r = NULL; // unreachable } return r; } diff --git a/hotspot/src/cpu/aarch64/vm/c1_MacroAssembler_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/c1_MacroAssembler_aarch64.hpp index fe41973b106..24389643ca0 100644 --- a/hotspot/src/cpu/aarch64/vm/c1_MacroAssembler_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/c1_MacroAssembler_aarch64.hpp @@ -27,6 +27,7 @@ #define CPU_AARCH64_VM_C1_MACROASSEMBLER_AARCH64_HPP using MacroAssembler::build_frame; +using MacroAssembler::null_check; // C1_MacroAssembler contains high-level macros for C1 diff --git a/hotspot/src/cpu/aarch64/vm/frame_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/frame_aarch64.cpp index 77664cba6b2..bce575a10af 100644 --- a/hotspot/src/cpu/aarch64/vm/frame_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/frame_aarch64.cpp @@ -433,11 +433,11 @@ frame frame::sender_for_interpreter_frame(RegisterMap* map) const { // This is the sp before any possible extension (adapter/locals). intptr_t* unextended_sp = interpreter_frame_sender_sp(); -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI if (map->update_map()) { update_map_with_saved_link(map, (intptr_t**) addr_at(link_offset)); } -#endif // COMPILER2 +#endif // COMPILER2 || INCLUDE_JVMCI return frame(sender_sp, unextended_sp, link(), sender_pc()); } diff --git a/hotspot/src/cpu/aarch64/vm/globalDefinitions_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/globalDefinitions_aarch64.hpp index d392ccf5e48..4c9d377ff09 100644 --- a/hotspot/src/cpu/aarch64/vm/globalDefinitions_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/globalDefinitions_aarch64.hpp @@ -28,6 +28,10 @@ const int StackAlignmentInBytes = 16; +// Indicates whether the C calling conventions require that +// 32-bit integer argument values are extended to 64 bits. +const bool CCallingConventionRequiresIntsAsLongs = false; + #define SUPPORTS_NATIVE_CX8 // The maximum B/BL offset range on AArch64 is 128MB. diff --git a/hotspot/src/cpu/aarch64/vm/globals_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/globals_aarch64.hpp index 8e5ed8c43f0..b6a58d9a371 100644 --- a/hotspot/src/cpu/aarch64/vm/globals_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/globals_aarch64.hpp @@ -40,14 +40,7 @@ define_pd_global(bool, ImplicitNullChecks, true); // Generate code for im define_pd_global(bool, TrapBasedNullChecks, false); define_pd_global(bool, UncommonNullCast, true); // Uncommon-trap NULLs past to check cast -// See 4827828 for this change. There is no globals_core_i486.hpp. I can't -// assign a different value for C2 without touching a number of files. Use -// #ifdef to minimize the change as it's late in Mantis. -- FIXME. -// c1 doesn't have this problem because the fix to 4858033 assures us -// the the vep is aligned at CodeEntryAlignment whereas c2 only aligns -// the uep and the vep doesn't get real alignment but just slops on by -// only assured that the entry instruction meets the 5 byte size requirement. -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI define_pd_global(intx, CodeEntryAlignment, 64); #else define_pd_global(intx, CodeEntryAlignment, 16); diff --git a/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.cpp index e70b4cce54a..59d06cb65ee 100644 --- a/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.cpp @@ -1054,13 +1054,39 @@ void InterpreterMacroAssembler::profile_virtual_call(Register receiver, bind(skip_receiver_profile); // The method data pointer needs to be updated to reflect the new target. +#if INCLUDE_JVMCI + if (MethodProfileWidth == 0) { + update_mdp_by_constant(mdp, in_bytes(VirtualCallData::virtual_call_data_size())); + } +#else // INCLUDE_JVMCI update_mdp_by_constant(mdp, in_bytes(VirtualCallData:: virtual_call_data_size())); +#endif // INCLUDE_JVMCI bind(profile_continue); } } +#if INCLUDE_JVMCI +void InterpreterMacroAssembler::profile_called_method(Register method, Register mdp, Register reg2) { + assert_different_registers(method, mdp, reg2); + if (ProfileInterpreter && MethodProfileWidth > 0) { + Label profile_continue; + + // If no method data exists, go to profile_continue. + test_method_data_pointer(mdp, profile_continue); + + Label done; + record_item_in_profile_helper(method, mdp, reg2, 0, done, MethodProfileWidth, + &VirtualCallData::method_offset, &VirtualCallData::method_count_offset, in_bytes(VirtualCallData::nonprofiled_receiver_count_offset())); + bind(done); + + update_mdp_by_constant(mdp, in_bytes(VirtualCallData::virtual_call_data_size())); + bind(profile_continue); + } +} +#endif // INCLUDE_JVMCI + // This routine creates a state machine for updating the multi-row // type profile at a virtual call site (or other type-sensitive bytecode). // The machine visits each row (of receiver/count) until the receiver type @@ -1080,14 +1106,36 @@ void InterpreterMacroAssembler::record_klass_in_profile_helper( if (is_virtual_call) { increment_mdp_data_at(mdp, in_bytes(CounterData::count_offset())); } - return; - } +#if INCLUDE_JVMCI + else if (EnableJVMCI) { + increment_mdp_data_at(mdp, in_bytes(ReceiverTypeData::nonprofiled_receiver_count_offset())); + } +#endif // INCLUDE_JVMCI + } else { + int non_profiled_offset = -1; + if (is_virtual_call) { + non_profiled_offset = in_bytes(CounterData::count_offset()); + } +#if INCLUDE_JVMCI + else if (EnableJVMCI) { + non_profiled_offset = in_bytes(ReceiverTypeData::nonprofiled_receiver_count_offset()); + } +#endif // INCLUDE_JVMCI - int last_row = VirtualCallData::row_limit() - 1; + record_item_in_profile_helper(receiver, mdp, reg2, 0, done, TypeProfileWidth, + &VirtualCallData::receiver_offset, &VirtualCallData::receiver_count_offset, non_profiled_offset); + } +} + +void InterpreterMacroAssembler::record_item_in_profile_helper(Register item, Register mdp, + Register reg2, int start_row, Label& done, int total_rows, + OffsetFunction item_offset_fn, OffsetFunction item_count_offset_fn, + int non_profiled_offset) { + int last_row = total_rows - 1; assert(start_row <= last_row, "must be work left to do"); - // Test this row for both the receiver and for null. + // Test this row for both the item and for null. // Take any of three different outcomes: - // 1. found receiver => increment count and goto done + // 1. found item => increment count and goto done // 2. found null => keep looking for case 1, maybe allocate this cell // 3. found something else => keep looking for cases 1 and 2 // Case 3 is handled by a recursive call. @@ -1095,55 +1143,56 @@ void InterpreterMacroAssembler::record_klass_in_profile_helper( Label next_test; bool test_for_null_also = (row == start_row); - // See if the receiver is receiver[n]. - int recvr_offset = in_bytes(VirtualCallData::receiver_offset(row)); - test_mdp_data_at(mdp, recvr_offset, receiver, + // See if the item is item[n]. + int item_offset = in_bytes(item_offset_fn(row)); + test_mdp_data_at(mdp, item_offset, item, (test_for_null_also ? reg2 : noreg), next_test); - // (Reg2 now contains the receiver from the CallData.) + // (Reg2 now contains the item from the CallData.) - // The receiver is receiver[n]. Increment count[n]. - int count_offset = in_bytes(VirtualCallData::receiver_count_offset(row)); + // The item is item[n]. Increment count[n]. + int count_offset = in_bytes(item_count_offset_fn(row)); increment_mdp_data_at(mdp, count_offset); b(done); bind(next_test); if (test_for_null_also) { Label found_null; - // Failed the equality check on receiver[n]... Test for null. + // Failed the equality check on item[n]... Test for null. if (start_row == last_row) { // The only thing left to do is handle the null case. - if (is_virtual_call) { + if (non_profiled_offset >= 0) { cbz(reg2, found_null); - // Receiver did not match any saved receiver and there is no empty row for it. + // Item did not match any saved item and there is no empty row for it. // Increment total counter to indicate polymorphic case. - increment_mdp_data_at(mdp, in_bytes(CounterData::count_offset())); + increment_mdp_data_at(mdp, non_profiled_offset); b(done); bind(found_null); } else { - cbz(reg2, done); + cbnz(reg2, done); } break; } // Since null is rare, make it be the branch-taken case. - cbz(reg2,found_null); + cbz(reg2, found_null); // Put all the "Case 3" tests here. - record_klass_in_profile_helper(receiver, mdp, reg2, start_row + 1, done, is_virtual_call); + record_item_in_profile_helper(item, mdp, reg2, start_row + 1, done, total_rows, + item_offset_fn, item_count_offset_fn, non_profiled_offset); - // Found a null. Keep searching for a matching receiver, + // Found a null. Keep searching for a matching item, // but remember that this is an empty (unused) slot. bind(found_null); } } - // In the fall-through case, we found no matching receiver, but we - // observed the receiver[start_row] is NULL. + // In the fall-through case, we found no matching item, but we + // observed the item[start_row] is NULL. - // Fill in the receiver field and increment the count. - int recvr_offset = in_bytes(VirtualCallData::receiver_offset(start_row)); - set_mdp_data_at(mdp, recvr_offset, receiver); - int count_offset = in_bytes(VirtualCallData::receiver_count_offset(start_row)); + // Fill in the item field and increment the count. + int item_offset = in_bytes(item_offset_fn(start_row)); + set_mdp_data_at(mdp, item_offset, item); + int count_offset = in_bytes(item_count_offset_fn(start_row)); mov(reg2, DataLayout::counter_increment); set_mdp_data_at(mdp, count_offset, reg2); if (start_row > 0) { @@ -1347,9 +1396,8 @@ void InterpreterMacroAssembler::notify_method_entry() { // the code to check if the event should be sent. if (JvmtiExport::can_post_interpreter_events()) { Label L; - ldr(r3, Address(rthread, JavaThread::interp_only_mode_offset())); - tst(r3, ~0); - br(Assembler::EQ, L); + ldrw(r3, Address(rthread, JavaThread::interp_only_mode_offset())); + cbzw(r3, L); call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::post_method_entry)); bind(L); diff --git a/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.hpp index 81bf0f7bd09..d56d17ab47a 100644 --- a/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/interp_masm_aarch64.hpp @@ -33,6 +33,7 @@ // This file specializes the assember with interpreter-specific macros +typedef ByteSize (*OffsetFunction)(uint); class InterpreterMacroAssembler: public MacroAssembler { protected: @@ -234,6 +235,10 @@ class InterpreterMacroAssembler: public MacroAssembler { void record_klass_in_profile_helper(Register receiver, Register mdp, Register reg2, int start_row, Label& done, bool is_virtual_call); + void record_item_in_profile_helper(Register item, Register mdp, + Register reg2, int start_row, Label& done, int total_rows, + OffsetFunction item_offset_fn, OffsetFunction item_count_offset_fn, + int non_profiled_offset); void update_mdp_by_offset(Register mdp_in, int offset_of_offset); void update_mdp_by_offset(Register mdp_in, Register reg, int offset_of_disp); @@ -247,6 +252,7 @@ class InterpreterMacroAssembler: public MacroAssembler { void profile_virtual_call(Register receiver, Register mdp, Register scratch2, bool receiver_can_be_null = false); + void profile_called_method(Register method, Register mdp, Register reg2) NOT_JVMCI_RETURN; void profile_ret(Register return_bci, Register mdp); void profile_null_seen(Register mdp); void profile_typecheck(Register mdp, Register klass, Register scratch); diff --git a/hotspot/src/cpu/aarch64/vm/interpreter_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/interpreter_aarch64.cpp index cdb6d5eecc4..7c9b9bef572 100644 --- a/hotspot/src/cpu/aarch64/vm/interpreter_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/interpreter_aarch64.cpp @@ -229,6 +229,7 @@ void TemplateInterpreterGenerator::generate_transcendental_entry(AbstractInterpr break; default: ShouldNotReachHere(); + fn = NULL; // unreachable } const int gpargs = 0, rtype = 3; __ mov(rscratch1, fn); diff --git a/hotspot/src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp index 14c8367f371..b2f21031f18 100644 --- a/hotspot/src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/jniFastGetField_aarch64.cpp @@ -61,6 +61,7 @@ address JNI_FastGetField::generate_fast_get_int_field0(BasicType type) { case T_FLOAT: name = "jni_fast_GetFloatField"; break; case T_DOUBLE: name = "jni_fast_GetDoubleField"; break; default: ShouldNotReachHere(); + name = NULL; // unreachable } ResourceMark rm; BufferBlob* blob = BufferBlob::create(name, BUFFER_SIZE); @@ -125,6 +126,7 @@ address JNI_FastGetField::generate_fast_get_int_field0(BasicType type) { case T_FLOAT: slow_case_addr = jni_GetFloatField_addr(); break; case T_DOUBLE: slow_case_addr = jni_GetDoubleField_addr(); break; default: ShouldNotReachHere(); + slow_case_addr = NULL; // unreachable } { diff --git a/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp index dea64b41bc6..c64f1277aab 100644 --- a/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp @@ -678,7 +678,7 @@ address MacroAssembler::trampoline_call(Address entry, CodeBuffer *cbuf) { if (cbuf) cbuf->set_insts_mark(); relocate(entry.rspec()); - if (Assembler::reachable_from_branch_at(pc(), entry.target())) { + if (!far_branches()) { bl(entry.target()); } else { bl(pc()); @@ -733,8 +733,8 @@ address MacroAssembler::emit_trampoline_stub(int insts_call_instruction_offset, return stub; } -address MacroAssembler::ic_call(address entry) { - RelocationHolder rh = virtual_call_Relocation::spec(pc()); +address MacroAssembler::ic_call(address entry, jint method_index) { + RelocationHolder rh = virtual_call_Relocation::spec(pc(), method_index); // address const_ptr = long_constant((jlong)Universe::non_oop_word()); // unsigned long offset; // ldr_constant(rscratch2, const_ptr); diff --git a/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.hpp index de6365c681b..99afc3b5522 100644 --- a/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/macroAssembler_aarch64.hpp @@ -410,7 +410,7 @@ class MacroAssembler: public Assembler { #define WRAP(INSN) \ void INSN(Register Rd, Register Rn, Register Rm, Register Ra) { \ - if ((VM_Version::cpu_cpuFeatures() & VM_Version::CPU_A53MAC) && Ra != zr) \ + if ((VM_Version::features() & VM_Version::CPU_A53MAC) && Ra != zr) \ nop(); \ Assembler::INSN(Rd, Rn, Rm, Ra); \ } @@ -480,6 +480,32 @@ public: orr(Vd, T, Vn, Vn); } +public: + + // Generalized Test Bit And Branch, including a "far" variety which + // spans more than 32KiB. + void tbr(Condition cond, Register Rt, int bitpos, Label &dest, bool far = false) { + assert(cond == EQ || cond == NE, "must be"); + + if (far) + cond = ~cond; + + void (Assembler::* branch)(Register Rt, int bitpos, Label &L); + if (cond == Assembler::EQ) + branch = &Assembler::tbz; + else + branch = &Assembler::tbnz; + + if (far) { + Label L; + (this->*branch)(Rt, bitpos, L); + b(dest); + bind(L); + } else { + (this->*branch)(Rt, bitpos, dest); + } + } + // macro instructions for accessing and updating floating point // status register // @@ -976,7 +1002,7 @@ public: } // Emit the CompiledIC call idiom - address ic_call(address entry); + address ic_call(address entry, jint method_index = 0); public: diff --git a/hotspot/src/cpu/aarch64/vm/nativeInst_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/nativeInst_aarch64.hpp index 60e26e08270..d6b0cb84298 100644 --- a/hotspot/src/cpu/aarch64/vm/nativeInst_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/nativeInst_aarch64.hpp @@ -62,7 +62,6 @@ class NativeInstruction VALUE_OBJ_CLASS_SPEC { inline bool is_jump_or_nop(); inline bool is_cond_jump(); bool is_safepoint_poll(); - inline bool is_mov_literal64(); bool is_movz(); bool is_movk(); bool is_sigill_zombie_not_entrant(); @@ -98,6 +97,14 @@ class NativeInstruction VALUE_OBJ_CLASS_SPEC { static bool is_ldr_literal_at(address instr); static bool is_ldrw_to_zr(address instr); + static bool is_call_at(address instr) { + const uint32_t insn = (*(uint32_t*)instr); + return (insn >> 26) == 0b100101; + } + bool is_call() { + return is_call_at(addr_at(0)); + } + static bool maybe_cpool_ref(address instr) { return is_adrp_at(instr) || is_ldr_literal_at(instr); } @@ -157,11 +164,6 @@ class NativeCall: public NativeInstruction { inline friend NativeCall* nativeCall_at(address address); inline friend NativeCall* nativeCall_before(address return_address); - static bool is_call_at(address instr) { - const uint32_t insn = (*(uint32_t*)instr); - return (insn >> 26) == 0b100101; - } - static bool is_call_before(address return_address) { return is_call_at(return_address - NativeCall::return_address_offset); } diff --git a/hotspot/src/cpu/aarch64/vm/relocInfo_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/relocInfo_aarch64.cpp index 3ab4f0b7b2c..40b53127487 100644 --- a/hotspot/src/cpu/aarch64/vm/relocInfo_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/relocInfo_aarch64.cpp @@ -59,14 +59,20 @@ void Relocation::pd_set_data_value(address x, intptr_t o, bool verify_only) { address Relocation::pd_call_destination(address orig_addr) { assert(is_call(), "should be a call here"); - if (is_call()) { + if (NativeCall::is_call_at(addr())) { address trampoline = nativeCall_at(addr())->get_trampoline(); if (trampoline) { return nativeCallTrampolineStub_at(trampoline)->destination(); } } if (orig_addr != NULL) { - return MacroAssembler::pd_call_destination(orig_addr); + address new_addr = MacroAssembler::pd_call_destination(orig_addr); + // If call is branch to self, don't try to relocate it, just leave it + // as branch to self. This happens during code generation if the code + // buffer expands. It will be relocated to the trampoline above once + // code generation is complete. + new_addr = (new_addr == orig_addr) ? addr() : new_addr; + return new_addr; } return MacroAssembler::pd_call_destination(addr()); } diff --git a/hotspot/src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp index bf677ce43dc..22e8eb04780 100644 --- a/hotspot/src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp @@ -39,10 +39,13 @@ #ifdef COMPILER1 #include "c1/c1_Runtime1.hpp" #endif -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI #include "adfiles/ad_aarch64.hpp" #include "opto/runtime.hpp" #endif +#if INCLUDE_JVMCI +#include "jvmci/jvmciJavaClasses.hpp" +#endif #ifdef BUILTIN_SIM #include "../../../../../../simulator/simulator.hpp" @@ -109,14 +112,14 @@ class RegisterSaver { }; OopMap* RegisterSaver::save_live_registers(MacroAssembler* masm, int additional_frame_words, int* total_frame_words, bool save_vectors) { -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI if (save_vectors) { // Save upper half of vector registers int vect_words = 32 * 8 / wordSize; additional_frame_words += vect_words; } #else - assert(!save_vectors, "vectors are generated only by C2"); + assert(!save_vectors, "vectors are generated only by C2 and JVMCI"); #endif int frame_size_in_bytes = round_to(additional_frame_words*wordSize + @@ -166,7 +169,7 @@ OopMap* RegisterSaver::save_live_registers(MacroAssembler* masm, int additional_ void RegisterSaver::restore_live_registers(MacroAssembler* masm, bool restore_vectors) { #ifndef COMPILER2 - assert(!restore_vectors, "vectors are generated only by C2"); + assert(!restore_vectors, "vectors are generated only by C2 and JVMCI"); #endif __ pop_CPU_state(restore_vectors); __ leave(); @@ -547,6 +550,18 @@ void SharedRuntime::gen_i2c_adapter(MacroAssembler *masm, // Pre-load the register-jump target early, to schedule it better. __ ldr(rscratch1, Address(rmethod, in_bytes(Method::from_compiled_offset()))); +#if INCLUDE_JVMCI + if (EnableJVMCI) { + // check if this call should be routed towards a specific entry point + __ ldr(rscratch2, Address(rthread, in_bytes(JavaThread::jvmci_alternate_call_target_offset()))); + Label no_alternative_target; + __ cbz(rscratch2, no_alternative_target); + __ mov(rscratch1, rscratch2); + __ str(zr, Address(rthread, in_bytes(JavaThread::jvmci_alternate_call_target_offset()))); + __ bind(no_alternative_target); + } +#endif // INCLUDE_JVMCI + // Now generate the shuffle code. for (int i = 0; i < total_args_passed; i++) { if (sig_bt[i] == T_VOID) { @@ -2237,7 +2252,13 @@ void SharedRuntime::generate_deopt_blob() { // Allocate space for the code ResourceMark rm; // Setup code generation tools - CodeBuffer buffer("deopt_blob", 2048, 1024); + int pad = 0; +#if INCLUDE_JVMCI + if (EnableJVMCI) { + pad += 512; // Increase the buffer size when compiling for JVMCI + } +#endif + CodeBuffer buffer("deopt_blob", 2048+pad, 1024); MacroAssembler* masm = new MacroAssembler(&buffer); int frame_size_in_words; OopMap* map = NULL; @@ -2294,6 +2315,12 @@ void SharedRuntime::generate_deopt_blob() { __ b(cont); int reexecute_offset = __ pc() - start; +#if defined(INCLUDE_JVMCI) && !defined(COMPILER1) + if (EnableJVMCI && UseJVMCICompiler) { + // JVMCI does not use this kind of deoptimization + __ should_not_reach_here(); + } +#endif // Reexecute case // return address is the pc describes what bci to do re-execute at @@ -2304,6 +2331,44 @@ void SharedRuntime::generate_deopt_blob() { __ movw(rcpool, Deoptimization::Unpack_reexecute); // callee-saved __ b(cont); +#if INCLUDE_JVMCI + Label after_fetch_unroll_info_call; + int implicit_exception_uncommon_trap_offset = 0; + int uncommon_trap_offset = 0; + + if (EnableJVMCI) { + implicit_exception_uncommon_trap_offset = __ pc() - start; + + __ ldr(lr, Address(rthread, in_bytes(JavaThread::jvmci_implicit_exception_pc_offset()))); + __ str(zr, Address(rthread, in_bytes(JavaThread::jvmci_implicit_exception_pc_offset()))); + + uncommon_trap_offset = __ pc() - start; + + // Save everything in sight. + RegisterSaver::save_live_registers(masm, 0, &frame_size_in_words); + // fetch_unroll_info needs to call last_java_frame() + Label retaddr; + __ set_last_Java_frame(sp, noreg, retaddr, rscratch1); + + __ ldrw(c_rarg1, Address(rthread, in_bytes(JavaThread::pending_deoptimization_offset()))); + __ movw(rscratch1, -1); + __ strw(rscratch1, Address(rthread, in_bytes(JavaThread::pending_deoptimization_offset()))); + + __ movw(rcpool, (int32_t)Deoptimization::Unpack_reexecute); + __ mov(c_rarg0, rthread); + __ lea(rscratch1, + RuntimeAddress(CAST_FROM_FN_PTR(address, + Deoptimization::uncommon_trap))); + __ blrt(rscratch1, 2, 0, MacroAssembler::ret_type_integral); + __ bind(retaddr); + oop_maps->add_gc_map( __ pc()-start, map->deep_copy()); + + __ reset_last_Java_frame(false, false); + + __ b(after_fetch_unroll_info_call); + } // EnableJVMCI +#endif // INCLUDE_JVMCI + int exception_offset = __ pc() - start; // Prolog for exception case @@ -2395,7 +2460,13 @@ void SharedRuntime::generate_deopt_blob() { __ reset_last_Java_frame(false, true); - // Load UnrollBlock* into rdi +#if INCLUDE_JVMCI + if (EnableJVMCI) { + __ bind(after_fetch_unroll_info_call); + } +#endif + + // Load UnrollBlock* into r5 __ mov(r5, r0); __ ldrw(rcpool, Address(r5, Deoptimization::UnrollBlock::unpack_kind_offset_in_bytes())); @@ -2547,7 +2618,12 @@ void SharedRuntime::generate_deopt_blob() { _deopt_blob = DeoptimizationBlob::create(&buffer, oop_maps, 0, exception_offset, reexecute_offset, frame_size_in_words); _deopt_blob->set_unpack_with_exception_in_tls_offset(exception_in_tls_offset); - +#if INCLUDE_JVMCI + if (EnableJVMCI) { + _deopt_blob->set_uncommon_trap_offset(uncommon_trap_offset); + _deopt_blob->set_implicit_exception_uncommon_trap_offset(implicit_exception_uncommon_trap_offset); + } +#endif #ifdef BUILTIN_SIM if (NotifySimulator) { unsigned char *base = _deopt_blob->code_begin(); @@ -2560,7 +2636,7 @@ uint SharedRuntime::out_preserve_stack_slots() { return 0; } -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI //------------------------------generate_uncommon_trap_blob-------------------- void SharedRuntime::generate_uncommon_trap_blob() { // Allocate space for the code @@ -2943,7 +3019,7 @@ RuntimeStub* SharedRuntime::generate_resolve_blob(address destination, const cha } -#ifdef COMPILER2 +#if defined(COMPILER2) || INCLUDE_JVMCI // This is here instead of runtime_x86_64.cpp because it uses SimpleRuntimeFrame // //------------------------------generate_exception_blob--------------------------- diff --git a/hotspot/src/cpu/aarch64/vm/stubGenerator_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/stubGenerator_aarch64.cpp index 369a484c6ec..32e92a99e7d 100644 --- a/hotspot/src/cpu/aarch64/vm/stubGenerator_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/stubGenerator_aarch64.cpp @@ -958,8 +958,8 @@ class StubGenerator: public StubCodeGenerator { const Register t0 = r3, t1 = r4; if (is_backwards) { - __ lea(s, Address(s, count, Address::uxtw(exact_log2(-step)))); - __ lea(d, Address(d, count, Address::uxtw(exact_log2(-step)))); + __ lea(s, Address(s, count, Address::lsl(exact_log2(-step)))); + __ lea(d, Address(d, count, Address::lsl(exact_log2(-step)))); } Label done, tail; @@ -1051,10 +1051,10 @@ class StubGenerator: public StubCodeGenerator { __ cmp(rscratch2, count); __ br(Assembler::HS, end); if (size == (size_t)wordSize) { - __ ldr(temp, Address(a, rscratch2, Address::uxtw(exact_log2(size)))); + __ ldr(temp, Address(a, rscratch2, Address::lsl(exact_log2(size)))); __ verify_oop(temp); } else { - __ ldrw(r16, Address(a, rscratch2, Address::uxtw(exact_log2(size)))); + __ ldrw(r16, Address(a, rscratch2, Address::lsl(exact_log2(size)))); __ decode_heap_oop(temp); // calls verify_oop } __ add(rscratch2, rscratch2, size); @@ -1087,12 +1087,14 @@ class StubGenerator: public StubCodeGenerator { __ align(CodeEntryAlignment); StubCodeMark mark(this, "StubRoutines", name); address start = __ pc(); + __ enter(); + if (entry != NULL) { *entry = __ pc(); // caller can pass a 64-bit byte count here (from Unsafe.copyMemory) BLOCK_COMMENT("Entry:"); } - __ enter(); + if (is_oop) { __ push(RegSet::of(d, count), sp); // no registers are destroyed by this call @@ -1104,10 +1106,11 @@ class StubGenerator: public StubCodeGenerator { if (VerifyOops) verify_oop_array(size, d, count, r16); __ sub(count, count, 1); // make an inclusive end pointer - __ lea(count, Address(d, count, Address::uxtw(exact_log2(size)))); + __ lea(count, Address(d, count, Address::lsl(exact_log2(size)))); gen_write_ref_array_post_barrier(d, count, rscratch1); } __ leave(); + __ mov(r0, zr); // return 0 __ ret(lr); #ifdef BUILTIN_SIM { @@ -1140,11 +1143,16 @@ class StubGenerator: public StubCodeGenerator { StubCodeMark mark(this, "StubRoutines", name); address start = __ pc(); + __ enter(); + if (entry != NULL) { + *entry = __ pc(); + // caller can pass a 64-bit byte count here (from Unsafe.copyMemory) + BLOCK_COMMENT("Entry:"); + } __ cmp(d, s); __ br(Assembler::LS, nooverlap_target); - __ enter(); if (is_oop) { __ push(RegSet::of(d, count), sp); // no registers are destroyed by this call @@ -1160,6 +1168,7 @@ class StubGenerator: public StubCodeGenerator { gen_write_ref_array_post_barrier(d, count, rscratch1); } __ leave(); + __ mov(r0, zr); // return 0 __ ret(lr); #ifdef BUILTIN_SIM { @@ -1559,7 +1568,29 @@ class StubGenerator: public StubCodeGenerator { Register dst_pos, // destination position (c_rarg3) Register length, Register temp, - Label& L_failed) { Unimplemented(); } + Label& L_failed) { + BLOCK_COMMENT("arraycopy_range_checks:"); + + assert_different_registers(rscratch1, temp); + + // if (src_pos + length > arrayOop(src)->length()) FAIL; + __ ldrw(rscratch1, Address(src, arrayOopDesc::length_offset_in_bytes())); + __ addw(temp, length, src_pos); + __ cmpw(temp, rscratch1); + __ br(Assembler::HI, L_failed); + + // if (dst_pos + length > arrayOop(dst)->length()) FAIL; + __ ldrw(rscratch1, Address(dst, arrayOopDesc::length_offset_in_bytes())); + __ addw(temp, length, dst_pos); + __ cmpw(temp, rscratch1); + __ br(Assembler::HI, L_failed); + + // Have to clean up high 32 bits of 'src_pos' and 'dst_pos'. + __ movw(src_pos, src_pos); + __ movw(dst_pos, dst_pos); + + BLOCK_COMMENT("arraycopy_range_checks done"); + } // These stubs get called from some dumb test routine. // I'll write them properly when they're called from @@ -1569,6 +1600,309 @@ class StubGenerator: public StubCodeGenerator { } + // + // Generate 'unsafe' array copy stub + // Though just as safe as the other stubs, it takes an unscaled + // size_t argument instead of an element count. + // + // Input: + // c_rarg0 - source array address + // c_rarg1 - destination array address + // c_rarg2 - byte count, treated as ssize_t, can be zero + // + // Examines the alignment of the operands and dispatches + // to a long, int, short, or byte copy loop. + // + address generate_unsafe_copy(const char *name, + address byte_copy_entry) { +#ifdef PRODUCT + return StubRoutines::_jbyte_arraycopy; +#else + __ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", name); + address start = __ pc(); + __ enter(); // required for proper stackwalking of RuntimeStub frame + // bump this on entry, not on exit: + __ lea(rscratch2, ExternalAddress((address)&SharedRuntime::_unsafe_array_copy_ctr)); + __ incrementw(Address(rscratch2)); + __ b(RuntimeAddress(byte_copy_entry)); + return start; +#endif + } + + // + // Generate generic array copy stubs + // + // Input: + // c_rarg0 - src oop + // c_rarg1 - src_pos (32-bits) + // c_rarg2 - dst oop + // c_rarg3 - dst_pos (32-bits) + // c_rarg4 - element count (32-bits) + // + // Output: + // r0 == 0 - success + // r0 == -1^K - failure, where K is partial transfer count + // + address generate_generic_copy(const char *name, + address byte_copy_entry, address short_copy_entry, + address int_copy_entry, address oop_copy_entry, + address long_copy_entry, address checkcast_copy_entry) { + + Label L_failed, L_failed_0, L_objArray; + Label L_copy_bytes, L_copy_shorts, L_copy_ints, L_copy_longs; + + // Input registers + const Register src = c_rarg0; // source array oop + const Register src_pos = c_rarg1; // source position + const Register dst = c_rarg2; // destination array oop + const Register dst_pos = c_rarg3; // destination position + const Register length = c_rarg4; + + StubCodeMark mark(this, "StubRoutines", name); + + __ align(CodeEntryAlignment); + address start = __ pc(); + + __ enter(); // required for proper stackwalking of RuntimeStub frame + + // bump this on entry, not on exit: + inc_counter_np(SharedRuntime::_generic_array_copy_ctr); + + //----------------------------------------------------------------------- + // Assembler stub will be used for this call to arraycopy + // if the following conditions are met: + // + // (1) src and dst must not be null. + // (2) src_pos must not be negative. + // (3) dst_pos must not be negative. + // (4) length must not be negative. + // (5) src klass and dst klass should be the same and not NULL. + // (6) src and dst should be arrays. + // (7) src_pos + length must not exceed length of src. + // (8) dst_pos + length must not exceed length of dst. + // + + // if (src == NULL) return -1; + __ cbz(src, L_failed); + + // if (src_pos < 0) return -1; + __ tbnz(src_pos, 31, L_failed); // i.e. sign bit set + + // if (dst == NULL) return -1; + __ cbz(dst, L_failed); + + // if (dst_pos < 0) return -1; + __ tbnz(dst_pos, 31, L_failed); // i.e. sign bit set + + // registers used as temp + const Register scratch_length = r16; // elements count to copy + const Register scratch_src_klass = r17; // array klass + const Register lh = r18; // layout helper + + // if (length < 0) return -1; + __ movw(scratch_length, length); // length (elements count, 32-bits value) + __ tbnz(scratch_length, 31, L_failed); // i.e. sign bit set + + __ load_klass(scratch_src_klass, src); +#ifdef ASSERT + // assert(src->klass() != NULL); + { + BLOCK_COMMENT("assert klasses not null {"); + Label L1, L2; + __ cbnz(scratch_src_klass, L2); // it is broken if klass is NULL + __ bind(L1); + __ stop("broken null klass"); + __ bind(L2); + __ load_klass(rscratch1, dst); + __ cbz(rscratch1, L1); // this would be broken also + BLOCK_COMMENT("} assert klasses not null done"); + } +#endif + + // Load layout helper (32-bits) + // + // |array_tag| | header_size | element_type | |log2_element_size| + // 32 30 24 16 8 2 0 + // + // array_tag: typeArray = 0x3, objArray = 0x2, non-array = 0x0 + // + + const int lh_offset = in_bytes(Klass::layout_helper_offset()); + + // Handle objArrays completely differently... + const jint objArray_lh = Klass::array_layout_helper(T_OBJECT); + __ ldrw(lh, Address(scratch_src_klass, lh_offset)); + __ movw(rscratch1, objArray_lh); + __ eorw(rscratch2, lh, rscratch1); + __ cbzw(rscratch2, L_objArray); + + // if (src->klass() != dst->klass()) return -1; + __ load_klass(rscratch2, dst); + __ eor(rscratch2, rscratch2, scratch_src_klass); + __ cbnz(rscratch2, L_failed); + + // if (!src->is_Array()) return -1; + __ tbz(lh, 31, L_failed); // i.e. (lh >= 0) + + // At this point, it is known to be a typeArray (array_tag 0x3). +#ifdef ASSERT + { + BLOCK_COMMENT("assert primitive array {"); + Label L; + __ movw(rscratch2, Klass::_lh_array_tag_type_value << Klass::_lh_array_tag_shift); + __ cmpw(lh, rscratch2); + __ br(Assembler::GE, L); + __ stop("must be a primitive array"); + __ bind(L); + BLOCK_COMMENT("} assert primitive array done"); + } +#endif + + arraycopy_range_checks(src, src_pos, dst, dst_pos, scratch_length, + rscratch2, L_failed); + + // TypeArrayKlass + // + // src_addr = (src + array_header_in_bytes()) + (src_pos << log2elemsize); + // dst_addr = (dst + array_header_in_bytes()) + (dst_pos << log2elemsize); + // + + const Register rscratch1_offset = rscratch1; // array offset + const Register r18_elsize = lh; // element size + + __ ubfx(rscratch1_offset, lh, Klass::_lh_header_size_shift, + exact_log2(Klass::_lh_header_size_mask+1)); // array_offset + __ add(src, src, rscratch1_offset); // src array offset + __ add(dst, dst, rscratch1_offset); // dst array offset + BLOCK_COMMENT("choose copy loop based on element size"); + + // next registers should be set before the jump to corresponding stub + const Register from = c_rarg0; // source array address + const Register to = c_rarg1; // destination array address + const Register count = c_rarg2; // elements count + + // 'from', 'to', 'count' registers should be set in such order + // since they are the same as 'src', 'src_pos', 'dst'. + + assert(Klass::_lh_log2_element_size_shift == 0, "fix this code"); + + // The possible values of elsize are 0-3, i.e. exact_log2(element + // size in bytes). We do a simple bitwise binary search. + __ BIND(L_copy_bytes); + __ tbnz(r18_elsize, 1, L_copy_ints); + __ tbnz(r18_elsize, 0, L_copy_shorts); + __ lea(from, Address(src, src_pos));// src_addr + __ lea(to, Address(dst, dst_pos));// dst_addr + __ movw(count, scratch_length); // length + __ b(RuntimeAddress(byte_copy_entry)); + + __ BIND(L_copy_shorts); + __ lea(from, Address(src, src_pos, Address::lsl(1)));// src_addr + __ lea(to, Address(dst, dst_pos, Address::lsl(1)));// dst_addr + __ movw(count, scratch_length); // length + __ b(RuntimeAddress(short_copy_entry)); + + __ BIND(L_copy_ints); + __ tbnz(r18_elsize, 0, L_copy_longs); + __ lea(from, Address(src, src_pos, Address::lsl(2)));// src_addr + __ lea(to, Address(dst, dst_pos, Address::lsl(2)));// dst_addr + __ movw(count, scratch_length); // length + __ b(RuntimeAddress(int_copy_entry)); + + __ BIND(L_copy_longs); +#ifdef ASSERT + { + BLOCK_COMMENT("assert long copy {"); + Label L; + __ andw(lh, lh, Klass::_lh_log2_element_size_mask); // lh -> r18_elsize + __ cmpw(r18_elsize, LogBytesPerLong); + __ br(Assembler::EQ, L); + __ stop("must be long copy, but elsize is wrong"); + __ bind(L); + BLOCK_COMMENT("} assert long copy done"); + } +#endif + __ lea(from, Address(src, src_pos, Address::lsl(3)));// src_addr + __ lea(to, Address(dst, dst_pos, Address::lsl(3)));// dst_addr + __ movw(count, scratch_length); // length + __ b(RuntimeAddress(long_copy_entry)); + + // ObjArrayKlass + __ BIND(L_objArray); + // live at this point: scratch_src_klass, scratch_length, src[_pos], dst[_pos] + + Label L_plain_copy, L_checkcast_copy; + // test array classes for subtyping + __ load_klass(r18, dst); + __ cmp(scratch_src_klass, r18); // usual case is exact equality + __ br(Assembler::NE, L_checkcast_copy); + + // Identically typed arrays can be copied without element-wise checks. + arraycopy_range_checks(src, src_pos, dst, dst_pos, scratch_length, + rscratch2, L_failed); + + __ lea(from, Address(src, src_pos, Address::lsl(3))); + __ add(from, from, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); + __ lea(to, Address(dst, dst_pos, Address::lsl(3))); + __ add(to, to, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); + __ movw(count, scratch_length); // length + __ BIND(L_plain_copy); + __ b(RuntimeAddress(oop_copy_entry)); + + __ BIND(L_checkcast_copy); + // live at this point: scratch_src_klass, scratch_length, r18 (dst_klass) + { + // Before looking at dst.length, make sure dst is also an objArray. + __ ldrw(rscratch1, Address(r18, lh_offset)); + __ movw(rscratch2, objArray_lh); + __ eorw(rscratch1, rscratch1, rscratch2); + __ cbnzw(rscratch1, L_failed); + + // It is safe to examine both src.length and dst.length. + arraycopy_range_checks(src, src_pos, dst, dst_pos, scratch_length, + r18, L_failed); + + const Register rscratch2_dst_klass = rscratch2; + __ load_klass(rscratch2_dst_klass, dst); // reload + + // Marshal the base address arguments now, freeing registers. + __ lea(from, Address(src, src_pos, Address::lsl(3))); + __ add(from, from, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); + __ lea(to, Address(dst, dst_pos, Address::lsl(3))); + __ add(to, to, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); + __ movw(count, length); // length (reloaded) + Register sco_temp = c_rarg3; // this register is free now + assert_different_registers(from, to, count, sco_temp, + rscratch2_dst_klass, scratch_src_klass); + // assert_clean_int(count, sco_temp); + + // Generate the type check. + const int sco_offset = in_bytes(Klass::super_check_offset_offset()); + __ ldrw(sco_temp, Address(rscratch2_dst_klass, sco_offset)); + // assert_clean_int(sco_temp, r18); + generate_type_check(scratch_src_klass, sco_temp, rscratch2_dst_klass, L_plain_copy); + + // Fetch destination element klass from the ObjArrayKlass header. + int ek_offset = in_bytes(ObjArrayKlass::element_klass_offset()); + __ ldr(rscratch2_dst_klass, Address(rscratch2_dst_klass, ek_offset)); + __ ldrw(sco_temp, Address(rscratch2_dst_klass, sco_offset)); + + // the checkcast_copy loop needs two extra arguments: + assert(c_rarg3 == sco_temp, "#3 already in place"); + // Set up arguments for checkcast_copy_entry. + __ mov(c_rarg4, rscratch2_dst_klass); // dst.klass.element_klass + __ b(RuntimeAddress(checkcast_copy_entry)); + } + + __ BIND(L_failed); + __ mov(r0, -1); + __ leave(); // required for proper stackwalking of RuntimeStub frame + __ ret(lr); + + return start; + } + void generate_arraycopy_stubs() { address entry; address entry_jbyte_arraycopy; @@ -1655,6 +1989,18 @@ class StubGenerator: public StubCodeGenerator { StubRoutines::_checkcast_arraycopy = generate_checkcast_copy("checkcast_arraycopy", &entry_checkcast_arraycopy); StubRoutines::_checkcast_arraycopy_uninit = generate_checkcast_copy("checkcast_arraycopy_uninit", NULL, /*dest_uninitialized*/true); + + StubRoutines::_unsafe_arraycopy = generate_unsafe_copy("unsafe_arraycopy", + entry_jbyte_arraycopy); + + StubRoutines::_generic_arraycopy = generate_generic_copy("generic_arraycopy", + entry_jbyte_arraycopy, + entry_jshort_arraycopy, + entry_jint_arraycopy, + entry_oop_arraycopy, + entry_jlong_arraycopy, + entry_checkcast_arraycopy); + } void generate_math_stubs() { Unimplemented(); } @@ -1973,7 +2319,7 @@ class StubGenerator: public StubCodeGenerator { // c_rarg4 - input length // // Output: - // x0 - input length + // r0 - input length // address generate_cipherBlockChaining_decryptAESCrypt() { assert(UseAES, "need AES instructions and misaligned SSE support"); diff --git a/hotspot/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp index 1edb46adf16..37176c78072 100644 --- a/hotspot/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp @@ -223,6 +223,19 @@ address TemplateInterpreterGenerator::generate_deopt_entry_for(TosState state, __ restore_constant_pool_cache(); __ get_method(rmethod); +#if INCLUDE_JVMCI + // Check if we need to take lock at entry of synchronized method. + if (UseJVMCICompiler) { + Label L; + __ ldr(rscratch1, Address(rthread, Thread::pending_exception_offset())); + __ cbz(rscratch1, L); + // Clear flag. + __ strb(zr, Address(rthread, JavaThread::pending_monitorenter_offset())); + // Take lock. + lock_method(); + __ bind(L); + } +#endif // handle exceptions { Label L; @@ -367,7 +380,7 @@ void TemplateInterpreterGenerator::generate_counter_incr( __ br(Assembler::LT, *profile_method_continue); // if no method data exists, go to profile_method - __ test_method_data_pointer(r0, *profile_method); + __ test_method_data_pointer(rscratch2, *profile_method); } { diff --git a/hotspot/src/cpu/aarch64/vm/vmStructs_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/vmStructs_aarch64.hpp index f7acc1cc7ec..66da9564512 100644 --- a/hotspot/src/cpu/aarch64/vm/vmStructs_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/vmStructs_aarch64.hpp @@ -1,5 +1,5 @@ /* - * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2014, Red Hat Inc. All rights reserved. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * @@ -30,16 +30,8 @@ // constants required by the Serviceability Agent. This file is // referenced by vmStructs.cpp. -#define VM_STRUCTS_CPU(nonstatic_field, static_field, unchecked_nonstatic_field, volatile_nonstatic_field, nonproduct_nonstatic_field, c2_nonstatic_field, unchecked_c1_static_field, unchecked_c2_static_field) \ - \ - /******************************/ \ - /* JavaCallWrapper */ \ - /******************************/ \ - /******************************/ \ - /* JavaFrameAnchor */ \ - /******************************/ \ - volatile_nonstatic_field(JavaFrameAnchor, _last_Java_fp, intptr_t*) - +#define VM_STRUCTS_CPU(nonstatic_field, static_field, unchecked_nonstatic_field, volatile_nonstatic_field, nonproduct_nonstatic_field, c2_nonstatic_field, unchecked_c1_static_field, unchecked_c2_static_field) \ + volatile_nonstatic_field(JavaFrameAnchor, _last_Java_fp, intptr_t*) #define VM_TYPES_CPU(declare_type, declare_toplevel_type, declare_oop_type, declare_integer_type, declare_unsigned_integer_type, declare_c1_toplevel_type, declare_c2_type, declare_c2_toplevel_type) diff --git a/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.cpp b/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.cpp index 37c0e5bbb63..d48f4ee3a25 100644 --- a/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.cpp +++ b/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.cpp @@ -67,8 +67,6 @@ int VM_Version::_model2; int VM_Version::_variant; int VM_Version::_revision; int VM_Version::_stepping; -int VM_Version::_cpuFeatures; -const char* VM_Version::_features_str = ""; static BufferBlob* stub_blob; static const int stub_size = 550; @@ -129,7 +127,7 @@ void VM_Version::get_processor_features() { char buf[512]; - _cpuFeatures = auxv; + _features = auxv; int cpu_lines = 0; if (FILE *f = fopen("/proc/cpuinfo", "r")) { @@ -154,12 +152,12 @@ void VM_Version::get_processor_features() { } // Enable vendor specific features - if (_cpu == CPU_CAVIUM && _variant == 0) _cpuFeatures |= CPU_DMB_ATOMICS; - if (_cpu == CPU_ARM && (_model == 0xd03 || _model2 == 0xd03)) _cpuFeatures |= CPU_A53MAC; + if (_cpu == CPU_CAVIUM && _variant == 0) _features |= CPU_DMB_ATOMICS; + if (_cpu == CPU_ARM && (_model == 0xd03 || _model2 == 0xd03)) _features |= CPU_A53MAC; // If an olde style /proc/cpuinfo (cpu_lines == 1) then if _model is an A57 (0xd07) // we assume the worst and assume we could be on a big little system and have // undisclosed A53 cores which we could be swapped to at any stage - if (_cpu == CPU_ARM && cpu_lines == 1 && _model == 0xd07) _cpuFeatures |= CPU_A53MAC; + if (_cpu == CPU_ARM && cpu_lines == 1 && _model == 0xd07) _features |= CPU_A53MAC; sprintf(buf, "0x%02x:0x%x:0x%03x:%d", _cpu, _variant, _model, _revision); if (_model2) sprintf(buf+strlen(buf), "(0x%03x)", _model2); @@ -169,7 +167,7 @@ void VM_Version::get_processor_features() { if (auxv & HWCAP_SHA1) strcat(buf, ", sha1"); if (auxv & HWCAP_SHA2) strcat(buf, ", sha256"); - _features_str = os::strdup(buf); + _features_string = os::strdup(buf); if (FLAG_IS_DEFAULT(UseCRC32)) { UseCRC32 = (auxv & HWCAP_CRC32) != 0; @@ -182,6 +180,11 @@ void VM_Version::get_processor_features() { FLAG_SET_DEFAULT(UseAdler32Intrinsics, true); } + if (UseVectorizedMismatchIntrinsic) { + warning("UseVectorizedMismatchIntrinsic specified, but not available on this CPU."); + FLAG_SET_DEFAULT(UseVectorizedMismatchIntrinsic, false); + } + if (auxv & HWCAP_AES) { UseAES = UseAES || FLAG_IS_DEFAULT(UseAES); UseAESIntrinsics = @@ -199,6 +202,11 @@ void VM_Version::get_processor_features() { } } + if (UseAESCTRIntrinsics) { + warning("AES/CTR intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + if (FLAG_IS_DEFAULT(UseCRC32Intrinsics)) { UseCRC32Intrinsics = true; } @@ -267,7 +275,7 @@ void VM_Version::get_processor_features() { } if (FLAG_IS_DEFAULT(UseBarriersForVolatile)) { - UseBarriersForVolatile = (_cpuFeatures & CPU_DMB_ATOMICS) != 0; + UseBarriersForVolatile = (_features & CPU_DMB_ATOMICS) != 0; } if (FLAG_IS_DEFAULT(UsePopCountInstruction)) { diff --git a/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.hpp b/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.hpp index f5dc414b694..7cb6fd7ce1a 100644 --- a/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.hpp +++ b/hotspot/src/cpu/aarch64/vm/vm_version_aarch64.hpp @@ -30,7 +30,8 @@ #include "runtime/vm_version.hpp" class VM_Version : public Abstract_VM_Version { -public: + friend class JVMCIVMStructs; + protected: static int _cpu; static int _model; @@ -38,9 +39,6 @@ protected: static int _variant; static int _revision; static int _stepping; - static int _cpuFeatures; // features returned by the "cpuid" instruction - // 0 if this instruction is not available - static const char* _features_str; static void get_processor_features(); @@ -52,7 +50,7 @@ public: static void assert_is_initialized() { } - enum { + enum Family { CPU_ARM = 'A', CPU_BROADCOM = 'B', CPU_CAVIUM = 'C', @@ -64,9 +62,9 @@ public: CPU_QUALCOM = 'Q', CPU_MARVELL = 'V', CPU_INTEL = 'i', - } cpuFamily; + }; - enum { + enum Feature_Flag { CPU_FP = (1<<0), CPU_ASIMD = (1<<1), CPU_EVTSTRM = (1<<2), @@ -77,16 +75,13 @@ public: CPU_CRC32 = (1<<7), CPU_A53MAC = (1 << 30), CPU_DMB_ATOMICS = (1 << 31), - } cpuFeatureFlags; + }; - static const char* cpu_features() { return _features_str; } static int cpu_family() { return _cpu; } static int cpu_model() { return _model; } static int cpu_model2() { return _model2; } static int cpu_variant() { return _variant; } static int cpu_revision() { return _revision; } - static int cpu_cpuFeatures() { return _cpuFeatures; } - }; #endif // CPU_AARCH64_VM_VM_VERSION_AARCH64_HPP diff --git a/hotspot/src/cpu/ppc/vm/assembler_ppc.cpp b/hotspot/src/cpu/ppc/vm/assembler_ppc.cpp index 56564ac7ea7..66e4f08c3d4 100644 --- a/hotspot/src/cpu/ppc/vm/assembler_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/assembler_ppc.cpp @@ -53,9 +53,6 @@ int AbstractAssembler::code_fill_byte() { return 0x00; // illegal instruction 0x00000000 } -void Assembler::print_instruction(int inst) { - Unimplemented(); -} // Patch instruction `inst' at offset `inst_pos' to refer to // `dest_pos' and return the resulting instruction. We should have @@ -484,7 +481,7 @@ int Assembler::add_const_optimized(Register d, Register s, long x, Register tmp, if (d != s) { mr(d, s); } return 0; } - if (return_simm16_rest) { + if (return_simm16_rest && (d == s)) { return xd; } addi(d, s, xd); diff --git a/hotspot/src/cpu/ppc/vm/assembler_ppc.hpp b/hotspot/src/cpu/ppc/vm/assembler_ppc.hpp index 6c7103aefa4..035bd4abf26 100644 --- a/hotspot/src/cpu/ppc/vm/assembler_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/assembler_ppc.hpp @@ -31,10 +31,37 @@ // Address is an abstraction used to represent a memory location // as used in assembler instructions. // PPC instructions grok either baseReg + indexReg or baseReg + disp. -// So far we do not use this as simplification by this class is low -// on PPC with its simple addressing mode. Use RegisterOrConstant to -// represent an offset. class Address VALUE_OBJ_CLASS_SPEC { + private: + Register _base; // Base register. + Register _index; // Index register. + intptr_t _disp; // Displacement. + + public: + Address(Register b, Register i, address d = 0) + : _base(b), _index(i), _disp((intptr_t)d) { + assert(i == noreg || d == 0, "can't have both"); + } + + Address(Register b, address d = 0) + : _base(b), _index(noreg), _disp((intptr_t)d) {} + + Address(Register b, intptr_t d) + : _base(b), _index(noreg), _disp(d) {} + + Address(Register b, RegisterOrConstant roc) + : _base(b), _index(noreg), _disp(0) { + if (roc.is_constant()) _disp = roc.as_constant(); else _index = roc.as_register(); + } + + Address() + : _base(noreg), _index(noreg), _disp(0) {} + + // accessors + Register base() const { return _base; } + Register index() const { return _index; } + int disp() const { return (int)_disp; } + bool is_const() const { return _base == noreg && _index == noreg; } }; class AddressLiteral VALUE_OBJ_CLASS_SPEC { @@ -164,10 +191,14 @@ struct FunctionDescriptor VALUE_OBJ_CLASS_SPEC { }; #endif + +// The PPC Assembler: Pure assembler doing NO optimizations on the +// instruction level; i.e., what you write is what you get. The +// Assembler is generating code into a CodeBuffer. + class Assembler : public AbstractAssembler { protected: // Displacement routines - static void print_instruction(int inst); static int patched_branch(int dest_pos, int inst, int inst_pos); static int branch_destination(int inst, int pos); @@ -839,41 +870,38 @@ class Assembler : public AbstractAssembler { enum Predict { pt = 1, pn = 0 }; // pt = predict taken - // instruction must start at passed address + // Instruction must start at passed address. static int instr_len(unsigned char *instr) { return BytesPerInstWord; } - // instruction must be left-justified in argument - static int instr_len(unsigned long instr) { return BytesPerInstWord; } - // longest instructions static int instr_maxlen() { return BytesPerInstWord; } // Test if x is within signed immediate range for nbits. static bool is_simm(int x, unsigned int nbits) { assert(0 < nbits && nbits < 32, "out of bounds"); - const int min = -( ((int)1) << nbits-1 ); - const int maxplus1 = ( ((int)1) << nbits-1 ); + const int min = -(((int)1) << nbits-1); + const int maxplus1 = (((int)1) << nbits-1); return min <= x && x < maxplus1; } static bool is_simm(jlong x, unsigned int nbits) { assert(0 < nbits && nbits < 64, "out of bounds"); - const jlong min = -( ((jlong)1) << nbits-1 ); - const jlong maxplus1 = ( ((jlong)1) << nbits-1 ); + const jlong min = -(((jlong)1) << nbits-1); + const jlong maxplus1 = (((jlong)1) << nbits-1); return min <= x && x < maxplus1; } - // Test if x is within unsigned immediate range for nbits + // Test if x is within unsigned immediate range for nbits. static bool is_uimm(int x, unsigned int nbits) { assert(0 < nbits && nbits < 32, "out of bounds"); - const int maxplus1 = ( ((int)1) << nbits ); - return 0 <= x && x < maxplus1; + const unsigned int maxplus1 = (((unsigned int)1) << nbits); + return (unsigned int)x < maxplus1; } static bool is_uimm(jlong x, unsigned int nbits) { assert(0 < nbits && nbits < 64, "out of bounds"); - const jlong maxplus1 = ( ((jlong)1) << nbits ); - return 0 <= x && x < maxplus1; + const julong maxplus1 = (((julong)1) << nbits); + return (julong)x < maxplus1; } protected: @@ -1196,6 +1224,8 @@ class Assembler : public AbstractAssembler { inline void mullw_( Register d, Register a, Register b); inline void mulhw( Register d, Register a, Register b); inline void mulhw_( Register d, Register a, Register b); + inline void mulhwu( Register d, Register a, Register b); + inline void mulhwu_(Register d, Register a, Register b); inline void mulhd( Register d, Register a, Register b); inline void mulhd_( Register d, Register a, Register b); inline void mulhdu( Register d, Register a, Register b); @@ -1376,8 +1406,11 @@ class Assembler : public AbstractAssembler { inline void orc( Register a, Register s, Register b); inline void orc_( Register a, Register s, Register b); inline void extsb( Register a, Register s); + inline void extsb_( Register a, Register s); inline void extsh( Register a, Register s); + inline void extsh_( Register a, Register s); inline void extsw( Register a, Register s); + inline void extsw_( Register a, Register s); // extended mnemonics inline void nop(); @@ -1767,6 +1800,8 @@ class Assembler : public AbstractAssembler { inline void smt_yield(); inline void smt_mdoio(); inline void smt_mdoom(); + // >= Power8 + inline void smt_miso(); // trap instructions inline void twi_0(Register a); // for load with acquire semantics use load+twi_0+isync (trap can't occur) @@ -2168,6 +2203,7 @@ class Assembler : public AbstractAssembler { inline void load_const(Register d, void* a, Register tmp = noreg); inline void load_const(Register d, Label& L, Register tmp = noreg); inline void load_const(Register d, AddressLiteral& a, Register tmp = noreg); + inline void load_const32(Register d, int i); // load signed int (patchable) // Load a 64 bit constant, optimized, not identifyable. // Tmp can be used to increase ILP. Set return_simm16_rest = true to get a diff --git a/hotspot/src/cpu/ppc/vm/assembler_ppc.inline.hpp b/hotspot/src/cpu/ppc/vm/assembler_ppc.inline.hpp index e860dac7d43..9248ae5cb6f 100644 --- a/hotspot/src/cpu/ppc/vm/assembler_ppc.inline.hpp +++ b/hotspot/src/cpu/ppc/vm/assembler_ppc.inline.hpp @@ -117,6 +117,8 @@ inline void Assembler::mullw( Register d, Register a, Register b) { emit_int32( inline void Assembler::mullw_( Register d, Register a, Register b) { emit_int32(MULLW_OPCODE | rt(d) | ra(a) | rb(b) | oe(0) | rc(1)); } inline void Assembler::mulhw( Register d, Register a, Register b) { emit_int32(MULHW_OPCODE | rt(d) | ra(a) | rb(b) | rc(0)); } inline void Assembler::mulhw_( Register d, Register a, Register b) { emit_int32(MULHW_OPCODE | rt(d) | ra(a) | rb(b) | rc(1)); } +inline void Assembler::mulhwu( Register d, Register a, Register b) { emit_int32(MULHWU_OPCODE | rt(d) | ra(a) | rb(b) | rc(0)); } +inline void Assembler::mulhwu_(Register d, Register a, Register b) { emit_int32(MULHWU_OPCODE | rt(d) | ra(a) | rb(b) | rc(1)); } inline void Assembler::mulhd( Register d, Register a, Register b) { emit_int32(MULHD_OPCODE | rt(d) | ra(a) | rb(b) | rc(0)); } inline void Assembler::mulhd_( Register d, Register a, Register b) { emit_int32(MULHD_OPCODE | rt(d) | ra(a) | rb(b) | rc(1)); } inline void Assembler::mulhdu( Register d, Register a, Register b) { emit_int32(MULHDU_OPCODE | rt(d) | ra(a) | rb(b) | rc(0)); } @@ -206,8 +208,11 @@ inline void Assembler::andc_( Register a, Register s, Register b) { emit_in inline void Assembler::orc( Register a, Register s, Register b) { emit_int32(ORC_OPCODE | rta(a) | rs(s) | rb(b) | rc(0)); } inline void Assembler::orc_( Register a, Register s, Register b) { emit_int32(ORC_OPCODE | rta(a) | rs(s) | rb(b) | rc(1)); } inline void Assembler::extsb( Register a, Register s) { emit_int32(EXTSB_OPCODE | rta(a) | rs(s) | rc(0)); } +inline void Assembler::extsb_( Register a, Register s) { emit_int32(EXTSB_OPCODE | rta(a) | rs(s) | rc(1)); } inline void Assembler::extsh( Register a, Register s) { emit_int32(EXTSH_OPCODE | rta(a) | rs(s) | rc(0)); } +inline void Assembler::extsh_( Register a, Register s) { emit_int32(EXTSH_OPCODE | rta(a) | rs(s) | rc(1)); } inline void Assembler::extsw( Register a, Register s) { emit_int32(EXTSW_OPCODE | rta(a) | rs(s) | rc(0)); } +inline void Assembler::extsw_( Register a, Register s) { emit_int32(EXTSW_OPCODE | rta(a) | rs(s) | rc(1)); } // extended mnemonics inline void Assembler::nop() { Assembler::ori(R0, R0, 0); } @@ -609,6 +614,8 @@ inline void Assembler::smt_prio_high() { Assembler::or_unchecked(R3, R3, inline void Assembler::smt_yield() { Assembler::or_unchecked(R27, R27, R27); } inline void Assembler::smt_mdoio() { Assembler::or_unchecked(R29, R29, R29); } inline void Assembler::smt_mdoom() { Assembler::or_unchecked(R30, R30, R30); } +// >= Power8 +inline void Assembler::smt_miso() { Assembler::or_unchecked(R26, R26, R26); } inline void Assembler::twi_0(Register a) { twi_unchecked(0, a, 0);} @@ -967,12 +974,15 @@ inline void Assembler::load_const(Register d, Label& L, Register tmp) { // Load a 64 bit constant encoded by an AddressLiteral. patchable. inline void Assembler::load_const(Register d, AddressLiteral& a, Register tmp) { - assert(d != R0, "R0 not allowed"); // First relocate (we don't change the offset in the RelocationHolder, // just pass a.rspec()), then delegate to load_const(Register, long). relocate(a.rspec()); load_const(d, (long)a.value(), tmp); } +inline void Assembler::load_const32(Register d, int i) { + lis(d, i >> 16); + ori(d, d, i & 0xFFFF); +} #endif // CPU_PPC_VM_ASSEMBLER_PPC_INLINE_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_CodeStubs_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_CodeStubs_ppc.cpp new file mode 100644 index 00000000000..a19c7b7de11 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_CodeStubs_ppc.cpp @@ -0,0 +1,527 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_CodeStubs.hpp" +#include "c1/c1_FrameMap.hpp" +#include "c1/c1_LIRAssembler.hpp" +#include "c1/c1_MacroAssembler.hpp" +#include "c1/c1_Runtime1.hpp" +#include "nativeInst_ppc.hpp" +#include "runtime/sharedRuntime.hpp" +#include "utilities/macros.hpp" +#include "vmreg_ppc.inline.hpp" +#if INCLUDE_ALL_GCS +#include "gc/g1/g1SATBCardTableModRefBS.hpp" +#endif // INCLUDE_ALL_GCS + +#define __ ce->masm()-> + + +RangeCheckStub::RangeCheckStub(CodeEmitInfo* info, LIR_Opr index, + bool throw_index_out_of_bounds_exception) + : _throw_index_out_of_bounds_exception(throw_index_out_of_bounds_exception) + , _index(index) { + assert(info != NULL, "must have info"); + _info = new CodeEmitInfo(info); +} + +void RangeCheckStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + if (_info->deoptimize_on_exception()) { + address a = Runtime1::entry_for(Runtime1::predicate_failed_trap_id); + // May be used by optimizations like LoopInvariantCodeMotion or RangeCheckEliminator. + DEBUG_ONLY( __ untested("RangeCheckStub: predicate_failed_trap_id"); ) + //__ load_const_optimized(R0, a); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(a)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + debug_only(__ illtrap()); + return; + } + + address stub = _throw_index_out_of_bounds_exception ? Runtime1::entry_for(Runtime1::throw_index_exception_id) + : Runtime1::entry_for(Runtime1::throw_range_check_failed_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + + Register index = R0; // pass in R0 + if (_index->is_register()) { + __ extsw(index, _index->as_register()); + } else { + __ load_const_optimized(index, _index->as_jint()); + } + + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + debug_only(__ illtrap()); +} + + +PredicateFailedStub::PredicateFailedStub(CodeEmitInfo* info) { + _info = new CodeEmitInfo(info); +} + +void PredicateFailedStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + address a = Runtime1::entry_for(Runtime1::predicate_failed_trap_id); + //__ load_const_optimized(R0, a); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(a)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + debug_only(__ illtrap()); +} + + +void CounterOverflowStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + // Parameter 1: bci + __ load_const_optimized(R0, _bci); + __ std(R0, -16, R1_SP); + + // Parameter 2: Method* + Metadata *m = _method->as_constant_ptr()->as_metadata(); + AddressLiteral md = __ constant_metadata_address(m); // Notify OOP recorder (don't need the relocation). + __ load_const_optimized(R0, md.value()); + __ std(R0, -8, R1_SP); + + address a = Runtime1::entry_for(Runtime1::counter_overflow_id); + //__ load_const_optimized(R0, a); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(a)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + + __ b(_continuation); +} + + +void DivByZeroStub::emit_code(LIR_Assembler* ce) { + if (_offset != -1) { + ce->compilation()->implicit_exception_table()->append(_offset, __ offset()); + } + __ bind(_entry); + address stub = Runtime1::entry_for(Runtime1::throw_div0_exception_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + debug_only(__ illtrap()); +} + + +void ImplicitNullCheckStub::emit_code(LIR_Assembler* ce) { + address a; + if (_info->deoptimize_on_exception()) { + // Deoptimize, do not throw the exception, because it is probably wrong to do it here. + a = Runtime1::entry_for(Runtime1::predicate_failed_trap_id); + } else { + a = Runtime1::entry_for(Runtime1::throw_null_pointer_exception_id); + } + + if (ImplicitNullChecks || TrapBasedNullChecks) { + ce->compilation()->implicit_exception_table()->append(_offset, __ offset()); + } + __ bind(_entry); + //__ load_const_optimized(R0, a); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(a)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + debug_only(__ illtrap()); +} + + +// Implementation of SimpleExceptionStub +void SimpleExceptionStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + address stub = Runtime1::entry_for(_stub); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + if (_obj->is_valid()) { __ mr_if_needed(/*tmp1 in do_CheckCast*/ R4_ARG2, _obj->as_register()); } + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + debug_only( __ illtrap(); ) +} + + +// Implementation of NewInstanceStub +NewInstanceStub::NewInstanceStub(LIR_Opr klass_reg, LIR_Opr result, ciInstanceKlass* klass, CodeEmitInfo* info, Runtime1::StubID stub_id) { + _result = result; + _klass = klass; + _klass_reg = klass_reg; + _info = new CodeEmitInfo(info); + assert(stub_id == Runtime1::new_instance_id || + stub_id == Runtime1::fast_new_instance_id || + stub_id == Runtime1::fast_new_instance_init_check_id, + "need new_instance id"); + _stub_id = stub_id; +} + +void NewInstanceStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + address entry = Runtime1::entry_for(_stub_id); + //__ load_const_optimized(R0, entry); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(entry)); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + __ b(_continuation); +} + + +// Implementation of NewTypeArrayStub +NewTypeArrayStub::NewTypeArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result, CodeEmitInfo* info) { + _klass_reg = klass_reg; + _length = length; + _result = result; + _info = new CodeEmitInfo(info); +} + +void NewTypeArrayStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + address entry = Runtime1::entry_for(Runtime1::new_type_array_id); + //__ load_const_optimized(R0, entry); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(entry)); + __ mr_if_needed(/*op->tmp1()->as_register()*/ R5_ARG3, _length->as_register()); // already sign-extended + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + __ b(_continuation); +} + + +// Implementation of NewObjectArrayStub +NewObjectArrayStub::NewObjectArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result, CodeEmitInfo* info) { + _klass_reg = klass_reg; + _length = length; + _result = result; + _info = new CodeEmitInfo(info); +} + +void NewObjectArrayStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + address entry = Runtime1::entry_for(Runtime1::new_object_array_id); + //__ load_const_optimized(R0, entry); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(entry)); + __ mr_if_needed(/*op->tmp1()->as_register()*/ R5_ARG3, _length->as_register()); // already sign-extended + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + __ b(_continuation); +} + + +// Implementation of MonitorAccessStubs +MonitorEnterStub::MonitorEnterStub(LIR_Opr obj_reg, LIR_Opr lock_reg, CodeEmitInfo* info) + : MonitorAccessStub(obj_reg, lock_reg) { + _info = new CodeEmitInfo(info); +} + +void MonitorEnterStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + address stub = Runtime1::entry_for(ce->compilation()->has_fpu_code() ? Runtime1::monitorenter_id : Runtime1::monitorenter_nofpu_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mr_if_needed(/*scratch_opr()->as_register()*/ R4_ARG2, _obj_reg->as_register()); + assert(_lock_reg->as_register() == R5_ARG3, ""); + __ mtctr(R0); + __ bctrl(); + ce->add_call_info_here(_info); + ce->verify_oop_map(_info); + __ b(_continuation); +} + +void MonitorExitStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + if (_compute_lock) { + ce->monitor_address(_monitor_ix, _lock_reg); + } + address stub = Runtime1::entry_for(ce->compilation()->has_fpu_code() ? Runtime1::monitorexit_id : Runtime1::monitorexit_nofpu_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + assert(_lock_reg->as_register() == R4_ARG2, ""); + __ mtctr(R0); + __ bctrl(); + __ b(_continuation); +} + + +// Implementation of patching: +// - Copy the code at given offset to an inlined buffer (first the bytes, then the number of bytes). +// - Replace original code with a call to the stub. +// At Runtime: +// - call to stub, jump to runtime +// - in runtime: preserve all registers (especially objects, i.e., source and destination object) +// - in runtime: after initializing class, restore original code, reexecute instruction + +int PatchingStub::_patch_info_offset = -(5 * BytesPerInstWord); + +void PatchingStub::align_patch_site(MacroAssembler* ) { + // Patch sites on ppc are always properly aligned. +} + +#ifdef ASSERT +inline void compare_with_patch_site(address template_start, address pc_start, int bytes_to_copy) { + address start = template_start; + for (int i = 0; i < bytes_to_copy; i++) { + address ptr = (address)(pc_start + i); + int a_byte = (*ptr) & 0xFF; + assert(a_byte == *start++, "should be the same code"); + } +} +#endif + +void PatchingStub::emit_code(LIR_Assembler* ce) { + // copy original code here + assert(NativeGeneralJump::instruction_size <= _bytes_to_copy && _bytes_to_copy <= 0xFF, + "not enough room for call"); + assert((_bytes_to_copy & 0x3) == 0, "must copy a multiple of four bytes"); + + Label call_patch; + + int being_initialized_entry = __ offset(); + + if (_id == load_klass_id) { + // Produce a copy of the load klass instruction for use by the being initialized case. + AddressLiteral addrlit((address)NULL, metadata_Relocation::spec(_index)); + __ load_const(_obj, addrlit, R0); + DEBUG_ONLY( compare_with_patch_site(__ code_section()->start() + being_initialized_entry, _pc_start, _bytes_to_copy); ) + } else if (_id == load_mirror_id || _id == load_appendix_id) { + // Produce a copy of the load mirror instruction for use by the being initialized case. + AddressLiteral addrlit((address)NULL, oop_Relocation::spec(_index)); + __ load_const(_obj, addrlit, R0); + DEBUG_ONLY( compare_with_patch_site(__ code_section()->start() + being_initialized_entry, _pc_start, _bytes_to_copy); ) + } else { + // Make a copy the code which is going to be patched. + for (int i = 0; i < _bytes_to_copy; i++) { + address ptr = (address)(_pc_start + i); + int a_byte = (*ptr) & 0xFF; + __ emit_int8 (a_byte); + } + } + + address end_of_patch = __ pc(); + int bytes_to_skip = 0; + if (_id == load_mirror_id) { + int offset = __ offset(); + __ block_comment(" being_initialized check"); + + // Static field accesses have special semantics while the class + // initializer is being run so we emit a test which can be used to + // check that this code is being executed by the initializing + // thread. + assert(_obj != noreg, "must be a valid register"); + assert(_index >= 0, "must have oop index"); + __ mr(R0, _obj); // spill + __ ld(_obj, java_lang_Class::klass_offset_in_bytes(), _obj); + __ ld(_obj, in_bytes(InstanceKlass::init_thread_offset()), _obj); + __ cmpd(CCR0, _obj, R16_thread); + __ mr(_obj, R0); // restore + __ bne(CCR0, call_patch); + + // Load_klass patches may execute the patched code before it's + // copied back into place so we need to jump back into the main + // code of the nmethod to continue execution. + __ b(_patch_site_continuation); + + // Make sure this extra code gets skipped. + bytes_to_skip += __ offset() - offset; + } + + // Now emit the patch record telling the runtime how to find the + // pieces of the patch. We only need 3 bytes but it has to be + // aligned as an instruction so emit 4 bytes. + int sizeof_patch_record = 4; + bytes_to_skip += sizeof_patch_record; + + // Emit the offsets needed to find the code to patch. + int being_initialized_entry_offset = __ offset() - being_initialized_entry + sizeof_patch_record; + + // Emit the patch record. We need to emit a full word, so emit an extra empty byte. + __ emit_int8(0); + __ emit_int8(being_initialized_entry_offset); + __ emit_int8(bytes_to_skip); + __ emit_int8(_bytes_to_copy); + address patch_info_pc = __ pc(); + assert(patch_info_pc - end_of_patch == bytes_to_skip, "incorrect patch info"); + + address entry = __ pc(); + NativeGeneralJump::insert_unconditional((address)_pc_start, entry); + address target = NULL; + relocInfo::relocType reloc_type = relocInfo::none; + switch (_id) { + case access_field_id: target = Runtime1::entry_for(Runtime1::access_field_patching_id); break; + case load_klass_id: target = Runtime1::entry_for(Runtime1::load_klass_patching_id); + reloc_type = relocInfo::metadata_type; break; + case load_mirror_id: target = Runtime1::entry_for(Runtime1::load_mirror_patching_id); + reloc_type = relocInfo::oop_type; break; + case load_appendix_id: target = Runtime1::entry_for(Runtime1::load_appendix_patching_id); + reloc_type = relocInfo::oop_type; break; + default: ShouldNotReachHere(); + } + __ bind(call_patch); + + __ block_comment("patch entry point"); + //__ load_const(R0, target); + mtctr + bctrl must have size -_patch_info_offset + __ load_const32(R0, MacroAssembler::offset_to_global_toc(target)); + __ add(R0, R29_TOC, R0); + __ mtctr(R0); + __ bctrl(); + assert(_patch_info_offset == (patch_info_pc - __ pc()), "must not change"); + ce->add_call_info_here(_info); + __ b(_patch_site_entry); + if (_id == load_klass_id || _id == load_mirror_id || _id == load_appendix_id) { + CodeSection* cs = __ code_section(); + address pc = (address)_pc_start; + RelocIterator iter(cs, pc, pc + 1); + relocInfo::change_reloc_info_for_address(&iter, (address) pc, reloc_type, relocInfo::none); + } +} + + +void DeoptimizeStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + address stub = Runtime1::entry_for(Runtime1::deoptimize_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + + __ load_const_optimized(R0, _trap_request); // Pass trap request in R0. + __ bctrl(); + ce->add_call_info_here(_info); + debug_only(__ illtrap()); +} + + +void ArrayCopyStub::emit_code(LIR_Assembler* ce) { + //---------------slow case: call to native----------------- + __ bind(_entry); + __ mr(R3_ARG1, src()->as_register()); + __ extsw(R4_ARG2, src_pos()->as_register()); + __ mr(R5_ARG3, dst()->as_register()); + __ extsw(R6_ARG4, dst_pos()->as_register()); + __ extsw(R7_ARG5, length()->as_register()); + + ce->emit_static_call_stub(); + + bool success = ce->emit_trampoline_stub_for_call(SharedRuntime::get_resolve_static_call_stub()); + if (!success) { return; } + + __ relocate(relocInfo::static_call_type); + // Note: At this point we do not have the address of the trampoline + // stub, and the entry point might be too far away for bl, so __ pc() + // serves as dummy and the bl will be patched later. + __ code()->set_insts_mark(); + __ bl(__ pc()); + ce->add_call_info_here(info()); + ce->verify_oop_map(info()); + +#ifndef PRODUCT + const address counter = (address)&Runtime1::_arraycopy_slowcase_cnt; + const Register tmp = R3, tmp2 = R4; + int simm16_offs = __ load_const_optimized(tmp, counter, tmp2, true); + __ lwz(tmp2, simm16_offs, tmp); + __ addi(tmp2, tmp2, 1); + __ stw(tmp2, simm16_offs, tmp); +#endif + + __ b(_continuation); +} + + +/////////////////////////////////////////////////////////////////////////////////// +#if INCLUDE_ALL_GCS + +void G1PreBarrierStub::emit_code(LIR_Assembler* ce) { + // At this point we know that marking is in progress. + // If do_load() is true then we have to emit the + // load of the previous value; otherwise it has already + // been loaded into _pre_val. + + __ bind(_entry); + + assert(pre_val()->is_register(), "Precondition."); + Register pre_val_reg = pre_val()->as_register(); + + if (do_load()) { + ce->mem2reg(addr(), pre_val(), T_OBJECT, patch_code(), info(), false /*wide*/, false /*unaligned*/); + } + + __ cmpdi(CCR0, pre_val_reg, 0); + __ bc_far_optimized(Assembler::bcondCRbiIs1, __ bi0(CCR0, Assembler::equal), _continuation); + + address stub = Runtime1::entry_for(Runtime1::Runtime1::g1_pre_barrier_slow_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ std(pre_val_reg, -8, R1_SP); // Pass pre_val on stack. + __ mtctr(R0); + __ bctrl(); + __ b(_continuation); +} + +void G1PostBarrierStub::emit_code(LIR_Assembler* ce) { + __ bind(_entry); + + assert(addr()->is_register(), "Precondition."); + assert(new_val()->is_register(), "Precondition."); + Register addr_reg = addr()->as_pointer_register(); + Register new_val_reg = new_val()->as_register(); + + __ cmpdi(CCR0, new_val_reg, 0); + __ bc_far_optimized(Assembler::bcondCRbiIs1, __ bi0(CCR0, Assembler::equal), _continuation); + + address stub = Runtime1::entry_for(Runtime1::Runtime1::g1_post_barrier_slow_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ mr(R0, addr_reg); // Pass addr in R0. + __ bctrl(); + __ b(_continuation); +} + +#endif // INCLUDE_ALL_GCS +/////////////////////////////////////////////////////////////////////////////////// + +#undef __ diff --git a/hotspot/src/cpu/ppc/vm/c1_Defs_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_Defs_ppc.hpp new file mode 100644 index 00000000000..73edcaba207 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_Defs_ppc.hpp @@ -0,0 +1,76 @@ +/* + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_DEFS_PPC_HPP +#define CPU_PPC_VM_C1_DEFS_PPC_HPP + +// Native word offsets from memory address. +enum { +#if defined(VM_LITTLE_ENDIAN) + pd_lo_word_offset_in_bytes = 0, + pd_hi_word_offset_in_bytes = BytesPerInt +#else + pd_lo_word_offset_in_bytes = BytesPerInt, + pd_hi_word_offset_in_bytes = 0 +#endif +}; + + +// Explicit rounding operations are not required to implement the strictFP mode. +enum { + pd_strict_fp_requires_explicit_rounding = false +}; + + +// registers +enum { + pd_nof_cpu_regs_frame_map = 32, // Number of registers used during code emission. + pd_nof_caller_save_cpu_regs_frame_map = 27, // Number of cpu registers killed by calls. (At least R3_ARG1 ... R10_ARG8, but using all like C2.) + pd_nof_cpu_regs_reg_alloc = 27, // Number of registers that are visible to register allocator. + pd_nof_cpu_regs_linearscan = 32, // Number of registers visible linear scan. + pd_first_callee_saved_reg = pd_nof_caller_save_cpu_regs_frame_map, + pd_last_callee_saved_reg = pd_nof_cpu_regs_reg_alloc - 1, + pd_first_cpu_reg = 0, + pd_last_cpu_reg = pd_nof_cpu_regs_reg_alloc - 1, + + pd_nof_fpu_regs_frame_map = 32, // Number of registers used during code emission. + pd_nof_caller_save_fpu_regs_frame_map = 32, // Number of fpu registers killed by calls. + pd_nof_fpu_regs_reg_alloc = 32, // Number of registers that are visible to register allocator. + pd_nof_fpu_regs_linearscan = 32, // Number of registers visible to linear scan. + pd_first_fpu_reg = pd_nof_cpu_regs_frame_map, + pd_last_fpu_reg = pd_nof_cpu_regs_frame_map + pd_nof_fpu_regs_reg_alloc - 1, + + pd_nof_xmm_regs_linearscan = 0, + pd_nof_caller_save_xmm_regs = 0, + pd_first_xmm_reg = -1, + pd_last_xmm_reg = -1 +}; + +// For debug info: a float value in a register is saved in single precision by runtime stubs. +enum { + pd_float_saved_as_double = true +}; + +#endif // CPU_PPC_VM_C1_DEFS_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_FpuStackSim_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_FpuStackSim_ppc.hpp new file mode 100644 index 00000000000..19f85034ba9 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_FpuStackSim_ppc.hpp @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2005, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_FPUSTACKSIM_PPC_HPP +#define CPU_PPC_VM_C1_FPUSTACKSIM_PPC_HPP + +// No FPU stack on PPC. +class FpuStackSim; + +#endif // CPU_PPC_VM_C1_FPUSTACKSIM_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.cpp new file mode 100644 index 00000000000..90afc7873ce --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.cpp @@ -0,0 +1,394 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_FrameMap.hpp" +#include "c1/c1_LIR.hpp" +#include "runtime/sharedRuntime.hpp" +#include "vmreg_ppc.inline.hpp" + + +const int FrameMap::pd_c_runtime_reserved_arg_size = 7; + + +LIR_Opr FrameMap::map_to_opr(BasicType type, VMRegPair* reg, bool outgoing) { + LIR_Opr opr = LIR_OprFact::illegalOpr; + VMReg r_1 = reg->first(); + VMReg r_2 = reg->second(); + if (r_1->is_stack()) { + // Convert stack slot to an SP offset. + // The calling convention does not count the SharedRuntime::out_preserve_stack_slots() value + // so we must add it in here. + int st_off = (r_1->reg2stack() + SharedRuntime::out_preserve_stack_slots()) * VMRegImpl::stack_slot_size; + opr = LIR_OprFact::address(new LIR_Address(SP_opr, st_off + STACK_BIAS, type)); + } else if (r_1->is_Register()) { + Register reg = r_1->as_Register(); + //if (outgoing) { + // assert(!reg->is_in(), "should be using I regs"); + //} else { + // assert(!reg->is_out(), "should be using O regs"); + //} + if (r_2->is_Register() && (type == T_LONG || type == T_DOUBLE)) { + opr = as_long_opr(reg); + } else if (type == T_OBJECT || type == T_ARRAY) { + opr = as_oop_opr(reg); + } else { + opr = as_opr(reg); + } + } else if (r_1->is_FloatRegister()) { + assert(type == T_DOUBLE || type == T_FLOAT, "wrong type"); + FloatRegister f = r_1->as_FloatRegister(); + if (type == T_DOUBLE) { + opr = as_double_opr(f); + } else { + opr = as_float_opr(f); + } + } + return opr; +} + +// FrameMap +//-------------------------------------------------------- + +FloatRegister FrameMap::_fpu_regs [FrameMap::nof_fpu_regs]; + +LIR_Opr FrameMap::R0_opr; +LIR_Opr FrameMap::R1_opr; +LIR_Opr FrameMap::R2_opr; +LIR_Opr FrameMap::R3_opr; +LIR_Opr FrameMap::R4_opr; +LIR_Opr FrameMap::R5_opr; +LIR_Opr FrameMap::R6_opr; +LIR_Opr FrameMap::R7_opr; +LIR_Opr FrameMap::R8_opr; +LIR_Opr FrameMap::R9_opr; +LIR_Opr FrameMap::R10_opr; +LIR_Opr FrameMap::R11_opr; +LIR_Opr FrameMap::R12_opr; +LIR_Opr FrameMap::R13_opr; +LIR_Opr FrameMap::R14_opr; +LIR_Opr FrameMap::R15_opr; +LIR_Opr FrameMap::R16_opr; +LIR_Opr FrameMap::R17_opr; +LIR_Opr FrameMap::R18_opr; +LIR_Opr FrameMap::R19_opr; +LIR_Opr FrameMap::R20_opr; +LIR_Opr FrameMap::R21_opr; +LIR_Opr FrameMap::R22_opr; +LIR_Opr FrameMap::R23_opr; +LIR_Opr FrameMap::R24_opr; +LIR_Opr FrameMap::R25_opr; +LIR_Opr FrameMap::R26_opr; +LIR_Opr FrameMap::R27_opr; +LIR_Opr FrameMap::R28_opr; +LIR_Opr FrameMap::R29_opr; +LIR_Opr FrameMap::R30_opr; +LIR_Opr FrameMap::R31_opr; + +LIR_Opr FrameMap::R0_oop_opr; +//LIR_Opr FrameMap::R1_oop_opr; +LIR_Opr FrameMap::R2_oop_opr; +LIR_Opr FrameMap::R3_oop_opr; +LIR_Opr FrameMap::R4_oop_opr; +LIR_Opr FrameMap::R5_oop_opr; +LIR_Opr FrameMap::R6_oop_opr; +LIR_Opr FrameMap::R7_oop_opr; +LIR_Opr FrameMap::R8_oop_opr; +LIR_Opr FrameMap::R9_oop_opr; +LIR_Opr FrameMap::R10_oop_opr; +LIR_Opr FrameMap::R11_oop_opr; +LIR_Opr FrameMap::R12_oop_opr; +//LIR_Opr FrameMap::R13_oop_opr; +LIR_Opr FrameMap::R14_oop_opr; +LIR_Opr FrameMap::R15_oop_opr; +//LIR_Opr FrameMap::R16_oop_opr; +LIR_Opr FrameMap::R17_oop_opr; +LIR_Opr FrameMap::R18_oop_opr; +LIR_Opr FrameMap::R19_oop_opr; +LIR_Opr FrameMap::R20_oop_opr; +LIR_Opr FrameMap::R21_oop_opr; +LIR_Opr FrameMap::R22_oop_opr; +LIR_Opr FrameMap::R23_oop_opr; +LIR_Opr FrameMap::R24_oop_opr; +LIR_Opr FrameMap::R25_oop_opr; +LIR_Opr FrameMap::R26_oop_opr; +LIR_Opr FrameMap::R27_oop_opr; +LIR_Opr FrameMap::R28_oop_opr; +//LIR_Opr FrameMap::R29_oop_opr; +LIR_Opr FrameMap::R30_oop_opr; +LIR_Opr FrameMap::R31_oop_opr; + +LIR_Opr FrameMap::R0_metadata_opr; +//LIR_Opr FrameMap::R1_metadata_opr; +LIR_Opr FrameMap::R2_metadata_opr; +LIR_Opr FrameMap::R3_metadata_opr; +LIR_Opr FrameMap::R4_metadata_opr; +LIR_Opr FrameMap::R5_metadata_opr; +LIR_Opr FrameMap::R6_metadata_opr; +LIR_Opr FrameMap::R7_metadata_opr; +LIR_Opr FrameMap::R8_metadata_opr; +LIR_Opr FrameMap::R9_metadata_opr; +LIR_Opr FrameMap::R10_metadata_opr; +LIR_Opr FrameMap::R11_metadata_opr; +LIR_Opr FrameMap::R12_metadata_opr; +//LIR_Opr FrameMap::R13_metadata_opr; +LIR_Opr FrameMap::R14_metadata_opr; +LIR_Opr FrameMap::R15_metadata_opr; +//LIR_Opr FrameMap::R16_metadata_opr; +LIR_Opr FrameMap::R17_metadata_opr; +LIR_Opr FrameMap::R18_metadata_opr; +LIR_Opr FrameMap::R19_metadata_opr; +LIR_Opr FrameMap::R20_metadata_opr; +LIR_Opr FrameMap::R21_metadata_opr; +LIR_Opr FrameMap::R22_metadata_opr; +LIR_Opr FrameMap::R23_metadata_opr; +LIR_Opr FrameMap::R24_metadata_opr; +LIR_Opr FrameMap::R25_metadata_opr; +LIR_Opr FrameMap::R26_metadata_opr; +LIR_Opr FrameMap::R27_metadata_opr; +LIR_Opr FrameMap::R28_metadata_opr; +//LIR_Opr FrameMap::R29_metadata_opr; +LIR_Opr FrameMap::R30_metadata_opr; +LIR_Opr FrameMap::R31_metadata_opr; + +LIR_Opr FrameMap::SP_opr; + +LIR_Opr FrameMap::R0_long_opr; +LIR_Opr FrameMap::R3_long_opr; + +LIR_Opr FrameMap::F1_opr; +LIR_Opr FrameMap::F1_double_opr; + +LIR_Opr FrameMap::_caller_save_cpu_regs[] = { 0, }; +LIR_Opr FrameMap::_caller_save_fpu_regs[] = { 0, }; + +FloatRegister FrameMap::nr2floatreg (int rnr) { + assert(_init_done, "tables not initialized"); + debug_only(fpu_range_check(rnr);) + return _fpu_regs[rnr]; +} + + +// Returns true if reg could be smashed by a callee. +bool FrameMap::is_caller_save_register (LIR_Opr reg) { + if (reg->is_single_fpu() || reg->is_double_fpu()) { return true; } + if (reg->is_double_cpu()) { + return is_caller_save_register(reg->as_register_lo()) || + is_caller_save_register(reg->as_register_hi()); + } + return is_caller_save_register(reg->as_register()); +} + + +bool FrameMap::is_caller_save_register (Register r) { + // not visible to allocator: R0: scratch, R1: SP + // r->encoding() < 2 + nof_caller_save_cpu_regs(); + return true; // Currently all regs are caller save. +} + + +void FrameMap::initialize() { + assert(!_init_done, "once"); + + int i = 0; + + // Put generally available registers at the beginning (allocated, saved for GC). + for (int j = 0; j < nof_cpu_regs; ++j) { + Register rj = as_Register(j); + if (reg_needs_save(rj)) { + map_register(i++, rj); + } + } + assert(i == nof_cpu_regs_reg_alloc, "number of allocated registers"); + + // The following registers are not normally available. + for (int j = 0; j < nof_cpu_regs; ++j) { + Register rj = as_Register(j); + if (!reg_needs_save(rj)) { + map_register(i++, rj); + } + } + assert(i == nof_cpu_regs, "number of CPU registers"); + + for (i = 0; i < nof_fpu_regs; i++) { + _fpu_regs[i] = as_FloatRegister(i); + } + + _init_done = true; + + R0_opr = as_opr(R0); + R1_opr = as_opr(R1); + R2_opr = as_opr(R2); + R3_opr = as_opr(R3); + R4_opr = as_opr(R4); + R5_opr = as_opr(R5); + R6_opr = as_opr(R6); + R7_opr = as_opr(R7); + R8_opr = as_opr(R8); + R9_opr = as_opr(R9); + R10_opr = as_opr(R10); + R11_opr = as_opr(R11); + R12_opr = as_opr(R12); + R13_opr = as_opr(R13); + R14_opr = as_opr(R14); + R15_opr = as_opr(R15); + R16_opr = as_opr(R16); + R17_opr = as_opr(R17); + R18_opr = as_opr(R18); + R19_opr = as_opr(R19); + R20_opr = as_opr(R20); + R21_opr = as_opr(R21); + R22_opr = as_opr(R22); + R23_opr = as_opr(R23); + R24_opr = as_opr(R24); + R25_opr = as_opr(R25); + R26_opr = as_opr(R26); + R27_opr = as_opr(R27); + R28_opr = as_opr(R28); + R29_opr = as_opr(R29); + R30_opr = as_opr(R30); + R31_opr = as_opr(R31); + + R0_oop_opr = as_oop_opr(R0); + //R1_oop_opr = as_oop_opr(R1); + R2_oop_opr = as_oop_opr(R2); + R3_oop_opr = as_oop_opr(R3); + R4_oop_opr = as_oop_opr(R4); + R5_oop_opr = as_oop_opr(R5); + R6_oop_opr = as_oop_opr(R6); + R7_oop_opr = as_oop_opr(R7); + R8_oop_opr = as_oop_opr(R8); + R9_oop_opr = as_oop_opr(R9); + R10_oop_opr = as_oop_opr(R10); + R11_oop_opr = as_oop_opr(R11); + R12_oop_opr = as_oop_opr(R12); + //R13_oop_opr = as_oop_opr(R13); + R14_oop_opr = as_oop_opr(R14); + R15_oop_opr = as_oop_opr(R15); + //R16_oop_opr = as_oop_opr(R16); + R17_oop_opr = as_oop_opr(R17); + R18_oop_opr = as_oop_opr(R18); + R19_oop_opr = as_oop_opr(R19); + R20_oop_opr = as_oop_opr(R20); + R21_oop_opr = as_oop_opr(R21); + R22_oop_opr = as_oop_opr(R22); + R23_oop_opr = as_oop_opr(R23); + R24_oop_opr = as_oop_opr(R24); + R25_oop_opr = as_oop_opr(R25); + R26_oop_opr = as_oop_opr(R26); + R27_oop_opr = as_oop_opr(R27); + R28_oop_opr = as_oop_opr(R28); + //R29_oop_opr = as_oop_opr(R29); + R30_oop_opr = as_oop_opr(R30); + R31_oop_opr = as_oop_opr(R31); + + R0_metadata_opr = as_metadata_opr(R0); + //R1_metadata_opr = as_metadata_opr(R1); + R2_metadata_opr = as_metadata_opr(R2); + R3_metadata_opr = as_metadata_opr(R3); + R4_metadata_opr = as_metadata_opr(R4); + R5_metadata_opr = as_metadata_opr(R5); + R6_metadata_opr = as_metadata_opr(R6); + R7_metadata_opr = as_metadata_opr(R7); + R8_metadata_opr = as_metadata_opr(R8); + R9_metadata_opr = as_metadata_opr(R9); + R10_metadata_opr = as_metadata_opr(R10); + R11_metadata_opr = as_metadata_opr(R11); + R12_metadata_opr = as_metadata_opr(R12); + //R13_metadata_opr = as_metadata_opr(R13); + R14_metadata_opr = as_metadata_opr(R14); + R15_metadata_opr = as_metadata_opr(R15); + //R16_metadata_opr = as_metadata_opr(R16); + R17_metadata_opr = as_metadata_opr(R17); + R18_metadata_opr = as_metadata_opr(R18); + R19_metadata_opr = as_metadata_opr(R19); + R20_metadata_opr = as_metadata_opr(R20); + R21_metadata_opr = as_metadata_opr(R21); + R22_metadata_opr = as_metadata_opr(R22); + R23_metadata_opr = as_metadata_opr(R23); + R24_metadata_opr = as_metadata_opr(R24); + R25_metadata_opr = as_metadata_opr(R25); + R26_metadata_opr = as_metadata_opr(R26); + R27_metadata_opr = as_metadata_opr(R27); + R28_metadata_opr = as_metadata_opr(R28); + //R29_metadata_opr = as_metadata_opr(R29); + R30_metadata_opr = as_metadata_opr(R30); + R31_metadata_opr = as_metadata_opr(R31); + + SP_opr = as_pointer_opr(R1_SP); + + R0_long_opr = LIR_OprFact::double_cpu(cpu_reg2rnr(R0), cpu_reg2rnr(R0)); + R3_long_opr = LIR_OprFact::double_cpu(cpu_reg2rnr(R3), cpu_reg2rnr(R3)); + + F1_opr = as_float_opr(F1); + F1_double_opr = as_double_opr(F1); + + // All the allocated cpu regs are caller saved. + for (int i = 0; i < max_nof_caller_save_cpu_regs; i++) { + _caller_save_cpu_regs[i] = LIR_OprFact::single_cpu(i); + } + + // All the fpu regs are caller saved. + for (int i = 0; i < nof_caller_save_fpu_regs; i++) { + _caller_save_fpu_regs[i] = LIR_OprFact::single_fpu(i); + } +} + + +Address FrameMap::make_new_address(ByteSize sp_offset) const { + return Address(R1_SP, STACK_BIAS + in_bytes(sp_offset)); +} + + +VMReg FrameMap::fpu_regname (int n) { + return as_FloatRegister(n)->as_VMReg(); +} + + +LIR_Opr FrameMap::stack_pointer() { + return SP_opr; +} + + +// JSR 292 +// On PPC64, there is no need to save the SP, because neither +// method handle intrinsics, nor compiled lambda forms modify it. +LIR_Opr FrameMap::method_handle_invoke_SP_save_opr() { + return LIR_OprFact::illegalOpr; +} + + +bool FrameMap::validate_frame() { + int max_offset = in_bytes(framesize_in_bytes()); + int java_index = 0; + for (int i = 0; i < _incoming_arguments->length(); i++) { + LIR_Opr opr = _incoming_arguments->at(i); + if (opr->is_stack()) { + max_offset = MAX2(_argument_locations->at(java_index), max_offset); + } + java_index += type2size[opr->type()]; + } + return Assembler::is_simm16(max_offset + STACK_BIAS); +} diff --git a/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.hpp new file mode 100644 index 00000000000..5eee5ffca1f --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_FrameMap_ppc.hpp @@ -0,0 +1,202 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_FRAMEMAP_PPC_HPP +#define CPU_PPC_VM_C1_FRAMEMAP_PPC_HPP + + public: + + enum { + nof_reg_args = 8, // Registers R3-R10 are available for parameter passing. + first_available_sp_in_frame = frame::jit_out_preserve_size, + frame_pad_in_bytes = 0 + }; + + static const int pd_c_runtime_reserved_arg_size; + + static LIR_Opr R0_opr; + static LIR_Opr R1_opr; + static LIR_Opr R2_opr; + static LIR_Opr R3_opr; + static LIR_Opr R4_opr; + static LIR_Opr R5_opr; + static LIR_Opr R6_opr; + static LIR_Opr R7_opr; + static LIR_Opr R8_opr; + static LIR_Opr R9_opr; + static LIR_Opr R10_opr; + static LIR_Opr R11_opr; + static LIR_Opr R12_opr; + static LIR_Opr R13_opr; + static LIR_Opr R14_opr; + static LIR_Opr R15_opr; + static LIR_Opr R16_opr; + static LIR_Opr R17_opr; + static LIR_Opr R18_opr; + static LIR_Opr R19_opr; + static LIR_Opr R20_opr; + static LIR_Opr R21_opr; + static LIR_Opr R22_opr; + static LIR_Opr R23_opr; + static LIR_Opr R24_opr; + static LIR_Opr R25_opr; + static LIR_Opr R26_opr; + static LIR_Opr R27_opr; + static LIR_Opr R28_opr; + static LIR_Opr R29_opr; + static LIR_Opr R30_opr; + static LIR_Opr R31_opr; + + static LIR_Opr R0_oop_opr; + //R1: Stack pointer. Not an oop. + static LIR_Opr R2_oop_opr; + static LIR_Opr R3_oop_opr; + static LIR_Opr R4_oop_opr; + static LIR_Opr R5_oop_opr; + static LIR_Opr R6_oop_opr; + static LIR_Opr R7_oop_opr; + static LIR_Opr R8_oop_opr; + static LIR_Opr R9_oop_opr; + static LIR_Opr R10_oop_opr; + static LIR_Opr R11_oop_opr; + static LIR_Opr R12_oop_opr; + //R13: System thread register. Not usable. + static LIR_Opr R14_oop_opr; + static LIR_Opr R15_oop_opr; + //R16: Java thread register. Not an oop. + static LIR_Opr R17_oop_opr; + static LIR_Opr R18_oop_opr; + static LIR_Opr R19_oop_opr; + static LIR_Opr R20_oop_opr; + static LIR_Opr R21_oop_opr; + static LIR_Opr R22_oop_opr; + static LIR_Opr R23_oop_opr; + static LIR_Opr R24_oop_opr; + static LIR_Opr R25_oop_opr; + static LIR_Opr R26_oop_opr; + static LIR_Opr R27_oop_opr; + static LIR_Opr R28_oop_opr; + static LIR_Opr R29_oop_opr; + //R29: TOC register. Not an oop. + static LIR_Opr R30_oop_opr; + static LIR_Opr R31_oop_opr; + + static LIR_Opr R0_metadata_opr; + //R1: Stack pointer. Not metadata. + static LIR_Opr R2_metadata_opr; + static LIR_Opr R3_metadata_opr; + static LIR_Opr R4_metadata_opr; + static LIR_Opr R5_metadata_opr; + static LIR_Opr R6_metadata_opr; + static LIR_Opr R7_metadata_opr; + static LIR_Opr R8_metadata_opr; + static LIR_Opr R9_metadata_opr; + static LIR_Opr R10_metadata_opr; + static LIR_Opr R11_metadata_opr; + static LIR_Opr R12_metadata_opr; + //R13: System thread register. Not usable. + static LIR_Opr R14_metadata_opr; + static LIR_Opr R15_metadata_opr; + //R16: Java thread register. Not metadata. + static LIR_Opr R17_metadata_opr; + static LIR_Opr R18_metadata_opr; + static LIR_Opr R19_metadata_opr; + static LIR_Opr R20_metadata_opr; + static LIR_Opr R21_metadata_opr; + static LIR_Opr R22_metadata_opr; + static LIR_Opr R23_metadata_opr; + static LIR_Opr R24_metadata_opr; + static LIR_Opr R25_metadata_opr; + static LIR_Opr R26_metadata_opr; + static LIR_Opr R27_metadata_opr; + static LIR_Opr R28_metadata_opr; + //R29: TOC register. Not metadata. + static LIR_Opr R30_metadata_opr; + static LIR_Opr R31_metadata_opr; + + static LIR_Opr SP_opr; + + static LIR_Opr R0_long_opr; + static LIR_Opr R3_long_opr; + + static LIR_Opr F1_opr; + static LIR_Opr F1_double_opr; + + private: + static FloatRegister _fpu_regs [nof_fpu_regs]; + + static LIR_Opr as_long_single_opr(Register r) { + return LIR_OprFact::double_cpu(cpu_reg2rnr(r), cpu_reg2rnr(r)); + } + static LIR_Opr as_long_pair_opr(Register r) { + return LIR_OprFact::double_cpu(cpu_reg2rnr(r->successor()), cpu_reg2rnr(r)); + } + + public: + +#ifdef _LP64 + static LIR_Opr as_long_opr(Register r) { + return as_long_single_opr(r); + } + static LIR_Opr as_pointer_opr(Register r) { + return as_long_single_opr(r); + } +#else + static LIR_Opr as_long_opr(Register r) { + Unimplemented(); return 0; +// return as_long_pair_opr(r); + } + static LIR_Opr as_pointer_opr(Register r) { + Unimplemented(); return 0; +// return as_opr(r); + } +#endif + static LIR_Opr as_float_opr(FloatRegister r) { + return LIR_OprFact::single_fpu(r->encoding()); + } + static LIR_Opr as_double_opr(FloatRegister r) { + return LIR_OprFact::double_fpu(r->encoding()); + } + + static FloatRegister nr2floatreg (int rnr); + + static VMReg fpu_regname (int n); + + static bool is_caller_save_register(LIR_Opr reg); + static bool is_caller_save_register(Register r); + + static int nof_caller_save_cpu_regs() { return pd_nof_caller_save_cpu_regs_frame_map; } + static int last_cpu_reg() { return pd_last_cpu_reg; } + + // Registers which need to be saved in the frames (e.g. for GC). + // Register usage: + // R0: scratch + // R1: sp + // R13: system thread id + // R16: java thread + // R29: global TOC + static bool reg_needs_save(Register r) { return r != R0 && r != R1 && r != R13 && r != R16 && r != R29; } + +#endif // CPU_PPC_VM_C1_FRAMEMAP_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.cpp new file mode 100644 index 00000000000..8a64bab4be4 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.cpp @@ -0,0 +1,3133 @@ +/* + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_Compilation.hpp" +#include "c1/c1_LIRAssembler.hpp" +#include "c1/c1_MacroAssembler.hpp" +#include "c1/c1_Runtime1.hpp" +#include "c1/c1_ValueStack.hpp" +#include "ci/ciArrayKlass.hpp" +#include "ci/ciInstance.hpp" +#include "gc/shared/collectedHeap.hpp" +#include "gc/shared/barrierSet.hpp" +#include "gc/shared/cardTableModRefBS.hpp" +#include "nativeInst_ppc.hpp" +#include "oops/objArrayKlass.hpp" +#include "runtime/sharedRuntime.hpp" + +#define __ _masm-> + + +const ConditionRegister LIR_Assembler::BOOL_RESULT = CCR5; + + +bool LIR_Assembler::is_small_constant(LIR_Opr opr) { + Unimplemented(); return false; // Currently not used on this platform. +} + + +LIR_Opr LIR_Assembler::receiverOpr() { + return FrameMap::R3_oop_opr; +} + + +LIR_Opr LIR_Assembler::osrBufferPointer() { + return FrameMap::R3_opr; +} + + +// This specifies the stack pointer decrement needed to build the frame. +int LIR_Assembler::initial_frame_size_in_bytes() const { + return in_bytes(frame_map()->framesize_in_bytes()); +} + + +// Inline cache check: the inline cached class is in inline_cache_reg; +// we fetch the class of the receiver and compare it with the cached class. +// If they do not match we jump to slow case. +int LIR_Assembler::check_icache() { + int offset = __ offset(); + __ inline_cache_check(R3_ARG1, R19_inline_cache_reg); + return offset; +} + + +void LIR_Assembler::osr_entry() { + // On-stack-replacement entry sequence: + // + // 1. Create a new compiled activation. + // 2. Initialize local variables in the compiled activation. The expression + // stack must be empty at the osr_bci; it is not initialized. + // 3. Jump to the continuation address in compiled code to resume execution. + + // OSR entry point + offsets()->set_value(CodeOffsets::OSR_Entry, code_offset()); + BlockBegin* osr_entry = compilation()->hir()->osr_entry(); + ValueStack* entry_state = osr_entry->end()->state(); + int number_of_locks = entry_state->locks_size(); + + // Create a frame for the compiled activation. + __ build_frame(initial_frame_size_in_bytes(), bang_size_in_bytes()); + + // OSR buffer is + // + // locals[nlocals-1..0] + // monitors[number_of_locks-1..0] + // + // Locals is a direct copy of the interpreter frame so in the osr buffer + // the first slot in the local array is the last local from the interpreter + // and the last slot is local[0] (receiver) from the interpreter. + // + // Similarly with locks. The first lock slot in the osr buffer is the nth lock + // from the interpreter frame, the nth lock slot in the osr buffer is 0th lock + // in the interpreter frame (the method lock if a sync method). + + // Initialize monitors in the compiled activation. + // R3: pointer to osr buffer + // + // All other registers are dead at this point and the locals will be + // copied into place by code emitted in the IR. + + Register OSR_buf = osrBufferPointer()->as_register(); + { assert(frame::interpreter_frame_monitor_size() == BasicObjectLock::size(), "adjust code below"); + int monitor_offset = BytesPerWord * method()->max_locals() + + (2 * BytesPerWord) * (number_of_locks - 1); + // SharedRuntime::OSR_migration_begin() packs BasicObjectLocks in + // the OSR buffer using 2 word entries: first the lock and then + // the oop. + for (int i = 0; i < number_of_locks; i++) { + int slot_offset = monitor_offset - ((i * 2) * BytesPerWord); +#ifdef ASSERT + // Verify the interpreter's monitor has a non-null object. + { + Label L; + __ ld(R0, slot_offset + 1*BytesPerWord, OSR_buf); + __ cmpdi(CCR0, R0, 0); + __ bne(CCR0, L); + __ stop("locked object is NULL"); + __ bind(L); + } +#endif // ASSERT + // Copy the lock field into the compiled activation. + Address ml = frame_map()->address_for_monitor_lock(i), + mo = frame_map()->address_for_monitor_object(i); + assert(ml.index() == noreg && mo.index() == noreg, "sanity"); + __ ld(R0, slot_offset + 0, OSR_buf); + __ std(R0, ml.disp(), ml.base()); + __ ld(R0, slot_offset + 1*BytesPerWord, OSR_buf); + __ std(R0, mo.disp(), mo.base()); + } + } +} + + +int LIR_Assembler::emit_exception_handler() { + // If the last instruction is a call (typically to do a throw which + // is coming at the end after block reordering) the return address + // must still point into the code area in order to avoid assertion + // failures when searching for the corresponding bci => add a nop + // (was bug 5/14/1999 - gri). + __ nop(); + + // Generate code for the exception handler. + address handler_base = __ start_a_stub(exception_handler_size); + + if (handler_base == NULL) { + // Not enough space left for the handler. + bailout("exception handler overflow"); + return -1; + } + + int offset = code_offset(); + address entry_point = CAST_FROM_FN_PTR(address, Runtime1::entry_for(Runtime1::handle_exception_from_callee_id)); + //__ load_const_optimized(R0, entry_point); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(entry_point)); + __ mtctr(R0); + __ bctr(); + + guarantee(code_offset() - offset <= exception_handler_size, "overflow"); + __ end_a_stub(); + + return offset; +} + + +// Emit the code to remove the frame from the stack in the exception +// unwind path. +int LIR_Assembler::emit_unwind_handler() { + _masm->block_comment("Unwind handler"); + + int offset = code_offset(); + bool preserve_exception = method()->is_synchronized() || compilation()->env()->dtrace_method_probes(); + const Register Rexception = R3 /*LIRGenerator::exceptionOopOpr()*/, Rexception_save = R31; + + // Fetch the exception from TLS and clear out exception related thread state. + __ ld(Rexception, in_bytes(JavaThread::exception_oop_offset()), R16_thread); + __ li(R0, 0); + __ std(R0, in_bytes(JavaThread::exception_oop_offset()), R16_thread); + __ std(R0, in_bytes(JavaThread::exception_pc_offset()), R16_thread); + + __ bind(_unwind_handler_entry); + __ verify_not_null_oop(Rexception); + if (preserve_exception) { __ mr(Rexception_save, Rexception); } + + // Perform needed unlocking + MonitorExitStub* stub = NULL; + if (method()->is_synchronized()) { + monitor_address(0, FrameMap::R4_opr); + stub = new MonitorExitStub(FrameMap::R4_opr, true, 0); + __ unlock_object(R5, R6, R4, *stub->entry()); + __ bind(*stub->continuation()); + } + + if (compilation()->env()->dtrace_method_probes()) { + Unimplemented(); + } + + // Dispatch to the unwind logic. + address unwind_stub = Runtime1::entry_for(Runtime1::unwind_exception_id); + //__ load_const_optimized(R0, unwind_stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(unwind_stub)); + if (preserve_exception) { __ mr(Rexception, Rexception_save); } + __ mtctr(R0); + __ bctr(); + + // Emit the slow path assembly. + if (stub != NULL) { + stub->emit_code(this); + } + + return offset; +} + + +int LIR_Assembler::emit_deopt_handler() { + // If the last instruction is a call (typically to do a throw which + // is coming at the end after block reordering) the return address + // must still point into the code area in order to avoid assertion + // failures when searching for the corresponding bci => add a nop + // (was bug 5/14/1999 - gri). + __ nop(); + + // Generate code for deopt handler. + address handler_base = __ start_a_stub(deopt_handler_size); + + if (handler_base == NULL) { + // Not enough space left for the handler. + bailout("deopt handler overflow"); + return -1; + } + + int offset = code_offset(); + __ bl64_patchable(SharedRuntime::deopt_blob()->unpack(), relocInfo::runtime_call_type); + + guarantee(code_offset() - offset <= deopt_handler_size, "overflow"); + __ end_a_stub(); + + return offset; +} + + +void LIR_Assembler::jobject2reg(jobject o, Register reg) { + if (o == NULL) { + __ li(reg, 0); + } else { + AddressLiteral addrlit = __ constant_oop_address(o); + __ load_const(reg, addrlit, (reg != R0) ? R0 : noreg); + } +} + + +void LIR_Assembler::jobject2reg_with_patching(Register reg, CodeEmitInfo *info) { + // Allocate a new index in table to hold the object once it's been patched. + int oop_index = __ oop_recorder()->allocate_oop_index(NULL); + PatchingStub* patch = new PatchingStub(_masm, patching_id(info), oop_index); + + AddressLiteral addrlit((address)NULL, oop_Relocation::spec(oop_index)); + __ load_const(reg, addrlit, R0); + + patching_epilog(patch, lir_patch_normal, reg, info); +} + + +void LIR_Assembler::metadata2reg(Metadata* o, Register reg) { + AddressLiteral md = __ constant_metadata_address(o); // Notify OOP recorder (don't need the relocation) + __ load_const_optimized(reg, md.value(), (reg != R0) ? R0 : noreg); +} + + +void LIR_Assembler::klass2reg_with_patching(Register reg, CodeEmitInfo *info) { + // Allocate a new index in table to hold the klass once it's been patched. + int index = __ oop_recorder()->allocate_metadata_index(NULL); + PatchingStub* patch = new PatchingStub(_masm, PatchingStub::load_klass_id, index); + + AddressLiteral addrlit((address)NULL, metadata_Relocation::spec(index)); + assert(addrlit.rspec().type() == relocInfo::metadata_type, "must be an metadata reloc"); + __ load_const(reg, addrlit, R0); + + patching_epilog(patch, lir_patch_normal, reg, info); +} + + +void LIR_Assembler::emit_op3(LIR_Op3* op) { + const bool is_int = op->result_opr()->is_single_cpu(); + Register Rdividend = is_int ? op->in_opr1()->as_register() : op->in_opr1()->as_register_lo(); + Register Rdivisor = noreg; + Register Rscratch = op->in_opr3()->as_register(); + Register Rresult = is_int ? op->result_opr()->as_register() : op->result_opr()->as_register_lo(); + long divisor = -1; + + if (op->in_opr2()->is_register()) { + Rdivisor = is_int ? op->in_opr2()->as_register() : op->in_opr2()->as_register_lo(); + } else { + divisor = is_int ? op->in_opr2()->as_constant_ptr()->as_jint() + : op->in_opr2()->as_constant_ptr()->as_jlong(); + } + + assert(Rdividend != Rscratch, ""); + assert(Rdivisor != Rscratch, ""); + assert(op->code() == lir_idiv || op->code() == lir_irem, "Must be irem or idiv"); + + if (Rdivisor == noreg) { + if (divisor == 1) { // stupid, but can happen + if (op->code() == lir_idiv) { + __ mr_if_needed(Rresult, Rdividend); + } else { + __ li(Rresult, 0); + } + + } else if (is_power_of_2(divisor)) { + // Convert division by a power of two into some shifts and logical operations. + int log2 = log2_intptr(divisor); + + // Round towards 0. + if (divisor == 2) { + if (is_int) { + __ srwi(Rscratch, Rdividend, 31); + } else { + __ srdi(Rscratch, Rdividend, 63); + } + } else { + if (is_int) { + __ srawi(Rscratch, Rdividend, 31); + } else { + __ sradi(Rscratch, Rdividend, 63); + } + __ clrldi(Rscratch, Rscratch, 64-log2); + } + __ add(Rscratch, Rdividend, Rscratch); + + if (op->code() == lir_idiv) { + if (is_int) { + __ srawi(Rresult, Rscratch, log2); + } else { + __ sradi(Rresult, Rscratch, log2); + } + } else { // lir_irem + __ clrrdi(Rscratch, Rscratch, log2); + __ sub(Rresult, Rdividend, Rscratch); + } + + } else if (divisor == -1) { + if (op->code() == lir_idiv) { + __ neg(Rresult, Rdividend); + } else { + __ li(Rresult, 0); + } + + } else { + __ load_const_optimized(Rscratch, divisor); + if (op->code() == lir_idiv) { + if (is_int) { + __ divw(Rresult, Rdividend, Rscratch); // Can't divide minint/-1. + } else { + __ divd(Rresult, Rdividend, Rscratch); // Can't divide minint/-1. + } + } else { + assert(Rscratch != R0, "need both"); + if (is_int) { + __ divw(R0, Rdividend, Rscratch); // Can't divide minint/-1. + __ mullw(Rscratch, R0, Rscratch); + } else { + __ divd(R0, Rdividend, Rscratch); // Can't divide minint/-1. + __ mulld(Rscratch, R0, Rscratch); + } + __ sub(Rresult, Rdividend, Rscratch); + } + + } + return; + } + + Label regular, done; + if (is_int) { + __ cmpwi(CCR0, Rdivisor, -1); + } else { + __ cmpdi(CCR0, Rdivisor, -1); + } + __ bne(CCR0, regular); + if (op->code() == lir_idiv) { + __ neg(Rresult, Rdividend); + __ b(done); + __ bind(regular); + if (is_int) { + __ divw(Rresult, Rdividend, Rdivisor); // Can't divide minint/-1. + } else { + __ divd(Rresult, Rdividend, Rdivisor); // Can't divide minint/-1. + } + } else { // lir_irem + __ li(Rresult, 0); + __ b(done); + __ bind(regular); + if (is_int) { + __ divw(Rscratch, Rdividend, Rdivisor); // Can't divide minint/-1. + __ mullw(Rscratch, Rscratch, Rdivisor); + } else { + __ divd(Rscratch, Rdividend, Rdivisor); // Can't divide minint/-1. + __ mulld(Rscratch, Rscratch, Rdivisor); + } + __ sub(Rresult, Rdividend, Rscratch); + } + __ bind(done); +} + + +void LIR_Assembler::emit_opBranch(LIR_OpBranch* op) { +#ifdef ASSERT + assert(op->block() == NULL || op->block()->label() == op->label(), "wrong label"); + if (op->block() != NULL) _branch_target_blocks.append(op->block()); + if (op->ublock() != NULL) _branch_target_blocks.append(op->ublock()); + assert(op->info() == NULL, "shouldn't have CodeEmitInfo"); +#endif + + Label *L = op->label(); + if (op->cond() == lir_cond_always) { + __ b(*L); + } else { + Label done; + bool is_unordered = false; + if (op->code() == lir_cond_float_branch) { + assert(op->ublock() != NULL, "must have unordered successor"); + is_unordered = true; + } else { + assert(op->code() == lir_branch, "just checking"); + } + + bool positive = false; + Assembler::Condition cond = Assembler::equal; + switch (op->cond()) { + case lir_cond_equal: positive = true ; cond = Assembler::equal ; is_unordered = false; break; + case lir_cond_notEqual: positive = false; cond = Assembler::equal ; is_unordered = false; break; + case lir_cond_less: positive = true ; cond = Assembler::less ; break; + case lir_cond_belowEqual: assert(op->code() != lir_cond_float_branch, ""); // fallthru + case lir_cond_lessEqual: positive = false; cond = Assembler::greater; break; + case lir_cond_greater: positive = true ; cond = Assembler::greater; break; + case lir_cond_aboveEqual: assert(op->code() != lir_cond_float_branch, ""); // fallthru + case lir_cond_greaterEqual: positive = false; cond = Assembler::less ; break; + default: ShouldNotReachHere(); + } + int bo = positive ? Assembler::bcondCRbiIs1 : Assembler::bcondCRbiIs0; + int bi = Assembler::bi0(BOOL_RESULT, cond); + if (is_unordered) { + if (positive) { + if (op->ublock() == op->block()) { + __ bc_far_optimized(Assembler::bcondCRbiIs1, __ bi0(BOOL_RESULT, Assembler::summary_overflow), *L); + } + } else { + if (op->ublock() != op->block()) { __ bso(BOOL_RESULT, done); } + } + } + __ bc_far_optimized(bo, bi, *L); + __ bind(done); + } +} + + +void LIR_Assembler::emit_opConvert(LIR_OpConvert* op) { + Bytecodes::Code code = op->bytecode(); + LIR_Opr src = op->in_opr(), + dst = op->result_opr(); + + switch(code) { + case Bytecodes::_i2l: { + __ extsw(dst->as_register_lo(), src->as_register()); + break; + } + case Bytecodes::_l2i: { + __ mr_if_needed(dst->as_register(), src->as_register_lo()); // high bits are garbage + break; + } + case Bytecodes::_i2b: { + __ extsb(dst->as_register(), src->as_register()); + break; + } + case Bytecodes::_i2c: { + __ clrldi(dst->as_register(), src->as_register(), 64-16); + break; + } + case Bytecodes::_i2s: { + __ extsh(dst->as_register(), src->as_register()); + break; + } + case Bytecodes::_i2d: + case Bytecodes::_l2d: { + __ fcfid(dst->as_double_reg(), src->as_double_reg()); // via mem + break; + } + case Bytecodes::_i2f: { + FloatRegister rdst = dst->as_float_reg(); + FloatRegister rsrc = src->as_double_reg(); // via mem + if (VM_Version::has_fcfids()) { + __ fcfids(rdst, rsrc); + } else { + __ fcfid(rdst, rsrc); + __ frsp(rdst, rdst); + } + break; + } + case Bytecodes::_l2f: { // >= Power7 + assert(VM_Version::has_fcfids(), "fcfid+frsp needs fixup code to avoid rounding incompatibility"); + __ fcfids(dst->as_float_reg(), src->as_double_reg()); // via mem + break; + } + case Bytecodes::_f2d: { + __ fmr_if_needed(dst->as_double_reg(), src->as_float_reg()); + break; + } + case Bytecodes::_d2f: { + __ frsp(dst->as_float_reg(), src->as_double_reg()); + break; + } + case Bytecodes::_d2i: + case Bytecodes::_f2i: { + FloatRegister rsrc = (code == Bytecodes::_d2i) ? src->as_double_reg() : src->as_float_reg(); + Address addr = frame_map()->address_for_slot(dst->double_stack_ix()); + Label L; + // Result must be 0 if value is NaN; test by comparing value to itself. + __ fcmpu(CCR0, rsrc, rsrc); + __ li(R0, 0); // 0 in case of NAN + __ std(R0, addr.disp(), addr.base()); + __ bso(CCR0, L); + __ fctiwz(rsrc, rsrc); // USE_KILL + __ stfd(rsrc, addr.disp(), addr.base()); + __ bind(L); + break; + } + case Bytecodes::_d2l: + case Bytecodes::_f2l: { + FloatRegister rsrc = (code == Bytecodes::_d2l) ? src->as_double_reg() : src->as_float_reg(); + Address addr = frame_map()->address_for_slot(dst->double_stack_ix()); + Label L; + // Result must be 0 if value is NaN; test by comparing value to itself. + __ fcmpu(CCR0, rsrc, rsrc); + __ li(R0, 0); // 0 in case of NAN + __ std(R0, addr.disp(), addr.base()); + __ bso(CCR0, L); + __ fctidz(rsrc, rsrc); // USE_KILL + __ stfd(rsrc, addr.disp(), addr.base()); + __ bind(L); + break; + } + + default: ShouldNotReachHere(); + } +} + + +void LIR_Assembler::align_call(LIR_Code) { + // do nothing since all instructions are word aligned on ppc +} + + +bool LIR_Assembler::emit_trampoline_stub_for_call(address target, Register Rtoc) { + int start_offset = __ offset(); + // Put the entry point as a constant into the constant pool. + const address entry_point_toc_addr = __ address_constant(target, RelocationHolder::none); + if (entry_point_toc_addr == NULL) { + bailout("const section overflow"); + return false; + } + const int entry_point_toc_offset = __ offset_to_method_toc(entry_point_toc_addr); + + // Emit the trampoline stub which will be related to the branch-and-link below. + address stub = __ emit_trampoline_stub(entry_point_toc_offset, start_offset, Rtoc); + if (!stub) { + bailout("no space for trampoline stub"); + return false; + } + return true; +} + + +void LIR_Assembler::call(LIR_OpJavaCall* op, relocInfo::relocType rtype) { + assert(rtype==relocInfo::opt_virtual_call_type || rtype==relocInfo::static_call_type, "unexpected rtype"); + + bool success = emit_trampoline_stub_for_call(op->addr()); + if (!success) { return; } + + __ relocate(rtype); + // Note: At this point we do not have the address of the trampoline + // stub, and the entry point might be too far away for bl, so __ pc() + // serves as dummy and the bl will be patched later. + __ code()->set_insts_mark(); + __ bl(__ pc()); + add_call_info(code_offset(), op->info()); +} + + +void LIR_Assembler::ic_call(LIR_OpJavaCall* op) { + __ calculate_address_from_global_toc(R2_TOC, __ method_toc()); + + // Virtual call relocation will point to ic load. + address virtual_call_meta_addr = __ pc(); + // Load a clear inline cache. + AddressLiteral empty_ic((address) Universe::non_oop_word()); + bool success = __ load_const_from_method_toc(R19_inline_cache_reg, empty_ic, R2_TOC); + if (!success) { + bailout("const section overflow"); + return; + } + // Call to fixup routine. Fixup routine uses ScopeDesc info + // to determine who we intended to call. + __ relocate(virtual_call_Relocation::spec(virtual_call_meta_addr)); + + success = emit_trampoline_stub_for_call(op->addr(), R2_TOC); + if (!success) { return; } + + // Note: At this point we do not have the address of the trampoline + // stub, and the entry point might be too far away for bl, so __ pc() + // serves as dummy and the bl will be patched later. + __ bl(__ pc()); + add_call_info(code_offset(), op->info()); +} + + +void LIR_Assembler::vtable_call(LIR_OpJavaCall* op) { + ShouldNotReachHere(); // ic_call is used instead. +} + + +void LIR_Assembler::explicit_null_check(Register addr, CodeEmitInfo* info) { + ImplicitNullCheckStub* stub = new ImplicitNullCheckStub(code_offset(), info); + __ null_check(addr, stub->entry()); + append_code_stub(stub); +} + + +// Attention: caller must encode oop if needed +int LIR_Assembler::store(LIR_Opr from_reg, Register base, int offset, BasicType type, bool wide, bool unaligned) { + int store_offset; + if (!Assembler::is_simm16(offset)) { + // For offsets larger than a simm16 we setup the offset. + assert(wide && !from_reg->is_same_register(FrameMap::R0_opr), "large offset only supported in special case"); + __ load_const_optimized(R0, offset); + store_offset = store(from_reg, base, R0, type, wide); + } else { + store_offset = code_offset(); + switch (type) { + case T_BOOLEAN: // fall through + case T_BYTE : __ stb(from_reg->as_register(), offset, base); break; + case T_CHAR : + case T_SHORT : __ sth(from_reg->as_register(), offset, base); break; + case T_INT : __ stw(from_reg->as_register(), offset, base); break; + case T_LONG : __ std(from_reg->as_register_lo(), offset, base); break; + case T_ADDRESS: + case T_METADATA: __ std(from_reg->as_register(), offset, base); break; + case T_ARRAY : // fall through + case T_OBJECT: + { + if (UseCompressedOops && !wide) { + // Encoding done in caller + __ stw(from_reg->as_register(), offset, base); + } else { + __ std(from_reg->as_register(), offset, base); + } + __ verify_oop(from_reg->as_register()); + break; + } + case T_FLOAT : __ stfs(from_reg->as_float_reg(), offset, base); break; + case T_DOUBLE: __ stfd(from_reg->as_double_reg(), offset, base); break; + default : ShouldNotReachHere(); + } + } + return store_offset; +} + + +// Attention: caller must encode oop if needed +int LIR_Assembler::store(LIR_Opr from_reg, Register base, Register disp, BasicType type, bool wide) { + int store_offset = code_offset(); + switch (type) { + case T_BOOLEAN: // fall through + case T_BYTE : __ stbx(from_reg->as_register(), base, disp); break; + case T_CHAR : + case T_SHORT : __ sthx(from_reg->as_register(), base, disp); break; + case T_INT : __ stwx(from_reg->as_register(), base, disp); break; + case T_LONG : +#ifdef _LP64 + __ stdx(from_reg->as_register_lo(), base, disp); +#else + Unimplemented(); +#endif + break; + case T_ADDRESS: + __ stdx(from_reg->as_register(), base, disp); + break; + case T_ARRAY : // fall through + case T_OBJECT: + { + if (UseCompressedOops && !wide) { + // Encoding done in caller. + __ stwx(from_reg->as_register(), base, disp); + } else { + __ stdx(from_reg->as_register(), base, disp); + } + __ verify_oop(from_reg->as_register()); // kills R0 + break; + } + case T_FLOAT : __ stfsx(from_reg->as_float_reg(), base, disp); break; + case T_DOUBLE: __ stfdx(from_reg->as_double_reg(), base, disp); break; + default : ShouldNotReachHere(); + } + return store_offset; +} + + +int LIR_Assembler::load(Register base, int offset, LIR_Opr to_reg, BasicType type, bool wide, bool unaligned) { + int load_offset; + if (!Assembler::is_simm16(offset)) { + // For offsets larger than a simm16 we setup the offset. + __ load_const_optimized(R0, offset); + load_offset = load(base, R0, to_reg, type, wide); + } else { + load_offset = code_offset(); + switch(type) { + case T_BOOLEAN: // fall through + case T_BYTE : __ lbz(to_reg->as_register(), offset, base); + __ extsb(to_reg->as_register(), to_reg->as_register()); break; + case T_CHAR : __ lhz(to_reg->as_register(), offset, base); break; + case T_SHORT : __ lha(to_reg->as_register(), offset, base); break; + case T_INT : __ lwa(to_reg->as_register(), offset, base); break; + case T_LONG : __ ld(to_reg->as_register_lo(), offset, base); break; + case T_METADATA: __ ld(to_reg->as_register(), offset, base); break; + case T_ADDRESS: + if (offset == oopDesc::klass_offset_in_bytes() && UseCompressedClassPointers) { + __ lwz(to_reg->as_register(), offset, base); + __ decode_klass_not_null(to_reg->as_register()); + } else { + __ ld(to_reg->as_register(), offset, base); + } + break; + case T_ARRAY : // fall through + case T_OBJECT: + { + if (UseCompressedOops && !wide) { + __ lwz(to_reg->as_register(), offset, base); + __ decode_heap_oop(to_reg->as_register()); + } else { + __ ld(to_reg->as_register(), offset, base); + } + __ verify_oop(to_reg->as_register()); + break; + } + case T_FLOAT: __ lfs(to_reg->as_float_reg(), offset, base); break; + case T_DOUBLE: __ lfd(to_reg->as_double_reg(), offset, base); break; + default : ShouldNotReachHere(); + } + } + return load_offset; +} + + +int LIR_Assembler::load(Register base, Register disp, LIR_Opr to_reg, BasicType type, bool wide) { + int load_offset = code_offset(); + switch(type) { + case T_BOOLEAN: // fall through + case T_BYTE : __ lbzx(to_reg->as_register(), base, disp); + __ extsb(to_reg->as_register(), to_reg->as_register()); break; + case T_CHAR : __ lhzx(to_reg->as_register(), base, disp); break; + case T_SHORT : __ lhax(to_reg->as_register(), base, disp); break; + case T_INT : __ lwax(to_reg->as_register(), base, disp); break; + case T_ADDRESS: __ ldx(to_reg->as_register(), base, disp); break; + case T_ARRAY : // fall through + case T_OBJECT: + { + if (UseCompressedOops && !wide) { + __ lwzx(to_reg->as_register(), base, disp); + __ decode_heap_oop(to_reg->as_register()); + } else { + __ ldx(to_reg->as_register(), base, disp); + } + __ verify_oop(to_reg->as_register()); + break; + } + case T_FLOAT: __ lfsx(to_reg->as_float_reg() , base, disp); break; + case T_DOUBLE: __ lfdx(to_reg->as_double_reg(), base, disp); break; + case T_LONG : +#ifdef _LP64 + __ ldx(to_reg->as_register_lo(), base, disp); +#else + Unimplemented(); +#endif + break; + default : ShouldNotReachHere(); + } + return load_offset; +} + + +void LIR_Assembler::const2stack(LIR_Opr src, LIR_Opr dest) { + LIR_Const* c = src->as_constant_ptr(); + Register src_reg = R0; + switch (c->type()) { + case T_INT: + case T_FLOAT: { + int value = c->as_jint_bits(); + __ load_const_optimized(src_reg, value); + Address addr = frame_map()->address_for_slot(dest->single_stack_ix()); + __ stw(src_reg, addr.disp(), addr.base()); + break; + } + case T_ADDRESS: { + int value = c->as_jint_bits(); + __ load_const_optimized(src_reg, value); + Address addr = frame_map()->address_for_slot(dest->single_stack_ix()); + __ std(src_reg, addr.disp(), addr.base()); + break; + } + case T_OBJECT: { + jobject2reg(c->as_jobject(), src_reg); + Address addr = frame_map()->address_for_slot(dest->single_stack_ix()); + __ std(src_reg, addr.disp(), addr.base()); + break; + } + case T_LONG: + case T_DOUBLE: { + int value = c->as_jlong_bits(); + __ load_const_optimized(src_reg, value); + Address addr = frame_map()->address_for_double_slot(dest->double_stack_ix()); + __ std(src_reg, addr.disp(), addr.base()); + break; + } + default: + Unimplemented(); + } +} + + +void LIR_Assembler::const2mem(LIR_Opr src, LIR_Opr dest, BasicType type, CodeEmitInfo* info, bool wide) { + LIR_Const* c = src->as_constant_ptr(); + LIR_Address* addr = dest->as_address_ptr(); + Register base = addr->base()->as_pointer_register(); + LIR_Opr tmp = LIR_OprFact::illegalOpr; + int offset = -1; + // Null check for large offsets in LIRGenerator::do_StoreField. + bool needs_explicit_null_check = !ImplicitNullChecks; + + if (info != NULL && needs_explicit_null_check) { + explicit_null_check(base, info); + } + + switch (c->type()) { + case T_FLOAT: type = T_INT; + case T_INT: + case T_ADDRESS: { + tmp = FrameMap::R0_opr; + __ load_const_optimized(tmp->as_register(), c->as_jint_bits()); + break; + } + case T_DOUBLE: type = T_LONG; + case T_LONG: { + tmp = FrameMap::R0_long_opr; + __ load_const_optimized(tmp->as_register_lo(), c->as_jlong_bits()); + break; + } + case T_OBJECT: { + tmp = FrameMap::R0_opr; + if (UseCompressedOops && !wide && c->as_jobject() != NULL) { + AddressLiteral oop_addr = __ constant_oop_address(c->as_jobject()); + __ lis(R0, oop_addr.value() >> 16); // Don't care about sign extend (will use stw). + __ relocate(oop_addr.rspec(), /*compressed format*/ 1); + __ ori(R0, R0, oop_addr.value() & 0xffff); + } else { + jobject2reg(c->as_jobject(), R0); + } + break; + } + default: + Unimplemented(); + } + + // Handle either reg+reg or reg+disp address. + if (addr->index()->is_valid()) { + assert(addr->disp() == 0, "must be zero"); + offset = store(tmp, base, addr->index()->as_pointer_register(), type, wide); + } else { + assert(Assembler::is_simm16(addr->disp()), "can't handle larger addresses"); + offset = store(tmp, base, addr->disp(), type, wide, false); + } + + if (info != NULL) { + assert(offset != -1, "offset should've been set"); + if (!needs_explicit_null_check) { + add_debug_info_for_null_check(offset, info); + } + } +} + + +void LIR_Assembler::const2reg(LIR_Opr src, LIR_Opr dest, LIR_PatchCode patch_code, CodeEmitInfo* info) { + LIR_Const* c = src->as_constant_ptr(); + LIR_Opr to_reg = dest; + + switch (c->type()) { + case T_INT: { + assert(patch_code == lir_patch_none, "no patching handled here"); + __ load_const_optimized(dest->as_register(), c->as_jint(), R0); + break; + } + case T_ADDRESS: { + assert(patch_code == lir_patch_none, "no patching handled here"); + __ load_const_optimized(dest->as_register(), c->as_jint(), R0); // Yes, as_jint ... + break; + } + case T_LONG: { + assert(patch_code == lir_patch_none, "no patching handled here"); + __ load_const_optimized(dest->as_register_lo(), c->as_jlong(), R0); + break; + } + + case T_OBJECT: { + if (patch_code == lir_patch_none) { + jobject2reg(c->as_jobject(), to_reg->as_register()); + } else { + jobject2reg_with_patching(to_reg->as_register(), info); + } + break; + } + + case T_METADATA: + { + if (patch_code == lir_patch_none) { + metadata2reg(c->as_metadata(), to_reg->as_register()); + } else { + klass2reg_with_patching(to_reg->as_register(), info); + } + } + break; + + case T_FLOAT: + { + if (to_reg->is_single_fpu()) { + address const_addr = __ float_constant(c->as_jfloat()); + if (const_addr == NULL) { + bailout("const section overflow"); + break; + } + RelocationHolder rspec = internal_word_Relocation::spec(const_addr); + __ relocate(rspec); + __ load_const(R0, const_addr); + __ lfsx(to_reg->as_float_reg(), R0); + } else { + assert(to_reg->is_single_cpu(), "Must be a cpu register."); + __ load_const_optimized(to_reg->as_register(), jint_cast(c->as_jfloat()), R0); + } + } + break; + + case T_DOUBLE: + { + if (to_reg->is_double_fpu()) { + address const_addr = __ double_constant(c->as_jdouble()); + if (const_addr == NULL) { + bailout("const section overflow"); + break; + } + RelocationHolder rspec = internal_word_Relocation::spec(const_addr); + __ relocate(rspec); + __ load_const(R0, const_addr); + __ lfdx(to_reg->as_double_reg(), R0); + } else { + assert(to_reg->is_double_cpu(), "Must be a long register."); + __ load_const_optimized(to_reg->as_register_lo(), jlong_cast(c->as_jdouble()), R0); + } + } + break; + + default: + ShouldNotReachHere(); + } +} + + +Address LIR_Assembler::as_Address(LIR_Address* addr) { + Unimplemented(); return Address(); +} + + +inline RegisterOrConstant index_or_disp(LIR_Address* addr) { + if (addr->index()->is_illegal()) { + return (RegisterOrConstant)(addr->disp()); + } else { + return (RegisterOrConstant)(addr->index()->as_pointer_register()); + } +} + + +void LIR_Assembler::stack2stack(LIR_Opr src, LIR_Opr dest, BasicType type) { + const Register tmp = R0; + switch (type) { + case T_INT: + case T_FLOAT: { + Address from = frame_map()->address_for_slot(src->single_stack_ix()); + Address to = frame_map()->address_for_slot(dest->single_stack_ix()); + __ lwz(tmp, from.disp(), from.base()); + __ stw(tmp, to.disp(), to.base()); + break; + } + case T_ADDRESS: + case T_OBJECT: { + Address from = frame_map()->address_for_slot(src->single_stack_ix()); + Address to = frame_map()->address_for_slot(dest->single_stack_ix()); + __ ld(tmp, from.disp(), from.base()); + __ std(tmp, to.disp(), to.base()); + break; + } + case T_LONG: + case T_DOUBLE: { + Address from = frame_map()->address_for_double_slot(src->double_stack_ix()); + Address to = frame_map()->address_for_double_slot(dest->double_stack_ix()); + __ ld(tmp, from.disp(), from.base()); + __ std(tmp, to.disp(), to.base()); + break; + } + + default: + ShouldNotReachHere(); + } +} + + +Address LIR_Assembler::as_Address_hi(LIR_Address* addr) { + Unimplemented(); return Address(); +} + + +Address LIR_Assembler::as_Address_lo(LIR_Address* addr) { + Unimplemented(); return Address(); +} + + +void LIR_Assembler::mem2reg(LIR_Opr src_opr, LIR_Opr dest, BasicType type, + LIR_PatchCode patch_code, CodeEmitInfo* info, bool wide, bool unaligned) { + + assert(type != T_METADATA, "load of metadata ptr not supported"); + LIR_Address* addr = src_opr->as_address_ptr(); + LIR_Opr to_reg = dest; + + Register src = addr->base()->as_pointer_register(); + Register disp_reg = noreg; + int disp_value = addr->disp(); + bool needs_patching = (patch_code != lir_patch_none); + // null check for large offsets in LIRGenerator::do_LoadField + bool needs_explicit_null_check = !os::zero_page_read_protected() || !ImplicitNullChecks; + + if (info != NULL && needs_explicit_null_check) { + explicit_null_check(src, info); + } + + if (addr->base()->type() == T_OBJECT) { + __ verify_oop(src); + } + + PatchingStub* patch = NULL; + if (needs_patching) { + patch = new PatchingStub(_masm, PatchingStub::access_field_id); + assert(!to_reg->is_double_cpu() || + patch_code == lir_patch_none || + patch_code == lir_patch_normal, "patching doesn't match register"); + } + + if (addr->index()->is_illegal()) { + if (!Assembler::is_simm16(disp_value)) { + if (needs_patching) { + __ load_const32(R0, 0); // patchable int + } else { + __ load_const_optimized(R0, disp_value); + } + disp_reg = R0; + } + } else { + disp_reg = addr->index()->as_pointer_register(); + assert(disp_value == 0, "can't handle 3 operand addresses"); + } + + // Remember the offset of the load. The patching_epilog must be done + // before the call to add_debug_info, otherwise the PcDescs don't get + // entered in increasing order. + int offset; + + if (disp_reg == noreg) { + assert(Assembler::is_simm16(disp_value), "should have set this up"); + offset = load(src, disp_value, to_reg, type, wide, unaligned); + } else { + assert(!unaligned, "unexpected"); + offset = load(src, disp_reg, to_reg, type, wide); + } + + if (patch != NULL) { + patching_epilog(patch, patch_code, src, info); + } + if (info != NULL && !needs_explicit_null_check) { + add_debug_info_for_null_check(offset, info); + } +} + + +void LIR_Assembler::stack2reg(LIR_Opr src, LIR_Opr dest, BasicType type) { + Address addr; + if (src->is_single_word()) { + addr = frame_map()->address_for_slot(src->single_stack_ix()); + } else if (src->is_double_word()) { + addr = frame_map()->address_for_double_slot(src->double_stack_ix()); + } + + bool unaligned = (addr.disp() - STACK_BIAS) % 8 != 0; + load(addr.base(), addr.disp(), dest, dest->type(), true /*wide*/, unaligned); +} + + +void LIR_Assembler::reg2stack(LIR_Opr from_reg, LIR_Opr dest, BasicType type, bool pop_fpu_stack) { + Address addr; + if (dest->is_single_word()) { + addr = frame_map()->address_for_slot(dest->single_stack_ix()); + } else if (dest->is_double_word()) { + addr = frame_map()->address_for_slot(dest->double_stack_ix()); + } + bool unaligned = (addr.disp() - STACK_BIAS) % 8 != 0; + store(from_reg, addr.base(), addr.disp(), from_reg->type(), true /*wide*/, unaligned); +} + + +void LIR_Assembler::reg2reg(LIR_Opr from_reg, LIR_Opr to_reg) { + if (from_reg->is_float_kind() && to_reg->is_float_kind()) { + if (from_reg->is_double_fpu()) { + // double to double moves + assert(to_reg->is_double_fpu(), "should match"); + __ fmr_if_needed(to_reg->as_double_reg(), from_reg->as_double_reg()); + } else { + // float to float moves + assert(to_reg->is_single_fpu(), "should match"); + __ fmr_if_needed(to_reg->as_float_reg(), from_reg->as_float_reg()); + } + } else if (!from_reg->is_float_kind() && !to_reg->is_float_kind()) { + if (from_reg->is_double_cpu()) { + __ mr_if_needed(to_reg->as_pointer_register(), from_reg->as_pointer_register()); + } else if (to_reg->is_double_cpu()) { + // int to int moves + __ mr_if_needed(to_reg->as_register_lo(), from_reg->as_register()); + } else { + // int to int moves + __ mr_if_needed(to_reg->as_register(), from_reg->as_register()); + } + } else { + ShouldNotReachHere(); + } + if (to_reg->type() == T_OBJECT || to_reg->type() == T_ARRAY) { + __ verify_oop(to_reg->as_register()); + } +} + + +void LIR_Assembler::reg2mem(LIR_Opr from_reg, LIR_Opr dest, BasicType type, + LIR_PatchCode patch_code, CodeEmitInfo* info, bool pop_fpu_stack, + bool wide, bool unaligned) { + assert(type != T_METADATA, "store of metadata ptr not supported"); + LIR_Address* addr = dest->as_address_ptr(); + + Register src = addr->base()->as_pointer_register(); + Register disp_reg = noreg; + int disp_value = addr->disp(); + bool needs_patching = (patch_code != lir_patch_none); + bool compress_oop = (type == T_ARRAY || type == T_OBJECT) && UseCompressedOops && !wide && + Universe::narrow_oop_mode() != Universe::UnscaledNarrowOop; + bool load_disp = addr->index()->is_illegal() && !Assembler::is_simm16(disp_value); + bool use_R29 = compress_oop && load_disp; // Avoid register conflict, also do null check before killing R29. + // Null check for large offsets in LIRGenerator::do_StoreField. + bool needs_explicit_null_check = !ImplicitNullChecks || use_R29; + + if (info != NULL && needs_explicit_null_check) { + explicit_null_check(src, info); + } + + if (addr->base()->is_oop_register()) { + __ verify_oop(src); + } + + PatchingStub* patch = NULL; + if (needs_patching) { + patch = new PatchingStub(_masm, PatchingStub::access_field_id); + assert(!from_reg->is_double_cpu() || + patch_code == lir_patch_none || + patch_code == lir_patch_normal, "patching doesn't match register"); + } + + if (addr->index()->is_illegal()) { + if (load_disp) { + disp_reg = use_R29 ? R29_TOC : R0; + if (needs_patching) { + __ load_const32(disp_reg, 0); // patchable int + } else { + __ load_const_optimized(disp_reg, disp_value); + } + } + } else { + disp_reg = addr->index()->as_pointer_register(); + assert(disp_value == 0, "can't handle 3 operand addresses"); + } + + // remember the offset of the store. The patching_epilog must be done + // before the call to add_debug_info_for_null_check, otherwise the PcDescs don't get + // entered in increasing order. + int offset; + + if (compress_oop) { + Register co = __ encode_heap_oop(R0, from_reg->as_register()); + from_reg = FrameMap::as_opr(co); + } + + if (disp_reg == noreg) { + assert(Assembler::is_simm16(disp_value), "should have set this up"); + offset = store(from_reg, src, disp_value, type, wide, unaligned); + } else { + assert(!unaligned, "unexpected"); + offset = store(from_reg, src, disp_reg, type, wide); + } + + if (use_R29) { + __ load_const_optimized(R29_TOC, MacroAssembler::global_toc(), R0); // reinit + } + + if (patch != NULL) { + patching_epilog(patch, patch_code, src, info); + } + + if (info != NULL && !needs_explicit_null_check) { + add_debug_info_for_null_check(offset, info); + } +} + + +void LIR_Assembler::return_op(LIR_Opr result) { + const Register return_pc = R11; + const Register polling_page = R12; + + // Pop the stack before the safepoint code. + int frame_size = initial_frame_size_in_bytes(); + if (Assembler::is_simm(frame_size, 16)) { + __ addi(R1_SP, R1_SP, frame_size); + } else { + __ pop_frame(); + } + + if (LoadPollAddressFromThread) { + // TODO: PPC port __ ld(polling_page, in_bytes(JavaThread::poll_address_offset()), R16_thread); + Unimplemented(); + } else { + __ load_const_optimized(polling_page, (long)(address) os::get_polling_page(), R0); // TODO: PPC port: get_standard_polling_page() + } + + // Restore return pc relative to callers' sp. + __ ld(return_pc, _abi(lr), R1_SP); + // Move return pc to LR. + __ mtlr(return_pc); + + // We need to mark the code position where the load from the safepoint + // polling page was emitted as relocInfo::poll_return_type here. + __ relocate(relocInfo::poll_return_type); + __ load_from_polling_page(polling_page); + + // Return. + __ blr(); +} + + +int LIR_Assembler::safepoint_poll(LIR_Opr tmp, CodeEmitInfo* info) { + + if (LoadPollAddressFromThread) { + const Register poll_addr = tmp->as_register(); + // TODO: PPC port __ ld(poll_addr, in_bytes(JavaThread::poll_address_offset()), R16_thread); + Unimplemented(); + __ relocate(relocInfo::poll_type); // XXX + guarantee(info != NULL, "Shouldn't be NULL"); + int offset = __ offset(); + add_debug_info_for_branch(info); + __ load_from_polling_page(poll_addr); + return offset; + } + + __ load_const_optimized(tmp->as_register(), (intptr_t)os::get_polling_page(), R0); // TODO: PPC port: get_standard_polling_page() + if (info != NULL) { + add_debug_info_for_branch(info); + } + int offset = __ offset(); + __ relocate(relocInfo::poll_type); + __ load_from_polling_page(tmp->as_register()); + + return offset; +} + + +void LIR_Assembler::emit_static_call_stub() { + address call_pc = __ pc(); + address stub = __ start_a_stub(max_static_call_stub_size); + if (stub == NULL) { + bailout("static call stub overflow"); + return; + } + + // For java_to_interp stubs we use R11_scratch1 as scratch register + // and in call trampoline stubs we use R12_scratch2. This way we + // can distinguish them (see is_NativeCallTrampolineStub_at()). + const Register reg_scratch = R11_scratch1; + + // Create a static stub relocation which relates this stub + // with the call instruction at insts_call_instruction_offset in the + // instructions code-section. + int start = __ offset(); + __ relocate(static_stub_Relocation::spec(call_pc)); + + // Now, create the stub's code: + // - load the TOC + // - load the inline cache oop from the constant pool + // - load the call target from the constant pool + // - call + __ calculate_address_from_global_toc(reg_scratch, __ method_toc()); + AddressLiteral ic = __ allocate_metadata_address((Metadata *)NULL); + bool success = __ load_const_from_method_toc(R19_inline_cache_reg, ic, reg_scratch, /*fixed_size*/ true); + + if (ReoptimizeCallSequences) { + __ b64_patchable((address)-1, relocInfo::none); + } else { + AddressLiteral a((address)-1); + success = success && __ load_const_from_method_toc(reg_scratch, a, reg_scratch, /*fixed_size*/ true); + __ mtctr(reg_scratch); + __ bctr(); + } + if (!success) { + bailout("const section overflow"); + return; + } + + assert(__ offset() - start <= max_static_call_stub_size, "stub too big"); + __ end_a_stub(); +} + + +void LIR_Assembler::comp_op(LIR_Condition condition, LIR_Opr opr1, LIR_Opr opr2, LIR_Op2* op) { + bool unsigned_comp = (condition == lir_cond_belowEqual || condition == lir_cond_aboveEqual); + if (opr1->is_single_fpu()) { + __ fcmpu(BOOL_RESULT, opr1->as_float_reg(), opr2->as_float_reg()); + } else if (opr1->is_double_fpu()) { + __ fcmpu(BOOL_RESULT, opr1->as_double_reg(), opr2->as_double_reg()); + } else if (opr1->is_single_cpu()) { + if (opr2->is_constant()) { + switch (opr2->as_constant_ptr()->type()) { + case T_INT: + { + jint con = opr2->as_constant_ptr()->as_jint(); + if (unsigned_comp) { + if (Assembler::is_uimm(con, 16)) { + __ cmplwi(BOOL_RESULT, opr1->as_register(), con); + } else { + __ load_const_optimized(R0, con); + __ cmplw(BOOL_RESULT, opr1->as_register(), R0); + } + } else { + if (Assembler::is_simm(con, 16)) { + __ cmpwi(BOOL_RESULT, opr1->as_register(), con); + } else { + __ load_const_optimized(R0, con); + __ cmpw(BOOL_RESULT, opr1->as_register(), R0); + } + } + } + break; + + case T_OBJECT: + // There are only equal/notequal comparisons on objects. + { + assert(condition == lir_cond_equal || condition == lir_cond_notEqual, "oops"); + jobject con = opr2->as_constant_ptr()->as_jobject(); + if (con == NULL) { + __ cmpdi(BOOL_RESULT, opr1->as_register(), 0); + } else { + jobject2reg(con, R0); + __ cmpd(BOOL_RESULT, opr1->as_register(), R0); + } + } + break; + + default: + ShouldNotReachHere(); + break; + } + } else { + if (opr2->is_address()) { + DEBUG_ONLY( Unimplemented(); ) // Seems to be unused at the moment. + LIR_Address *addr = opr2->as_address_ptr(); + BasicType type = addr->type(); + if (type == T_OBJECT) { __ ld(R0, index_or_disp(addr), addr->base()->as_register()); } + else { __ lwa(R0, index_or_disp(addr), addr->base()->as_register()); } + __ cmpd(BOOL_RESULT, opr1->as_register(), R0); + } else { + if (unsigned_comp) { + __ cmplw(BOOL_RESULT, opr1->as_register(), opr2->as_register()); + } else { + __ cmpw(BOOL_RESULT, opr1->as_register(), opr2->as_register()); + } + } + } + } else if (opr1->is_double_cpu()) { + if (opr2->is_constant()) { + jlong con = opr2->as_constant_ptr()->as_jlong(); + if (unsigned_comp) { + if (Assembler::is_uimm(con, 16)) { + __ cmpldi(BOOL_RESULT, opr1->as_register_lo(), con); + } else { + __ load_const_optimized(R0, con); + __ cmpld(BOOL_RESULT, opr1->as_register_lo(), R0); + } + } else { + if (Assembler::is_simm(con, 16)) { + __ cmpdi(BOOL_RESULT, opr1->as_register_lo(), con); + } else { + __ load_const_optimized(R0, con); + __ cmpd(BOOL_RESULT, opr1->as_register_lo(), R0); + } + } + } else if (opr2->is_register()) { + if (unsigned_comp) { + __ cmpld(BOOL_RESULT, opr1->as_register_lo(), opr2->as_register_lo()); + } else { + __ cmpd(BOOL_RESULT, opr1->as_register_lo(), opr2->as_register_lo()); + } + } else { + ShouldNotReachHere(); + } + } else if (opr1->is_address()) { + DEBUG_ONLY( Unimplemented(); ) // Seems to be unused at the moment. + LIR_Address * addr = opr1->as_address_ptr(); + BasicType type = addr->type(); + assert (opr2->is_constant(), "Checking"); + if (type == T_OBJECT) { __ ld(R0, index_or_disp(addr), addr->base()->as_register()); } + else { __ lwa(R0, index_or_disp(addr), addr->base()->as_register()); } + __ cmpdi(BOOL_RESULT, R0, opr2->as_constant_ptr()->as_jint()); + } else { + ShouldNotReachHere(); + } +} + + +void LIR_Assembler::comp_fl2i(LIR_Code code, LIR_Opr left, LIR_Opr right, LIR_Opr dst, LIR_Op2* op){ + const Register Rdst = dst->as_register(); + Label done; + if (code == lir_cmp_fd2i || code == lir_ucmp_fd2i) { + bool is_unordered_less = (code == lir_ucmp_fd2i); + if (left->is_single_fpu()) { + __ fcmpu(CCR0, left->as_float_reg(), right->as_float_reg()); + } else if (left->is_double_fpu()) { + __ fcmpu(CCR0, left->as_double_reg(), right->as_double_reg()); + } else { + ShouldNotReachHere(); + } + __ li(Rdst, is_unordered_less ? -1 : 1); + __ bso(CCR0, done); + } else if (code == lir_cmp_l2i) { + __ cmpd(CCR0, left->as_register_lo(), right->as_register_lo()); + } else { + ShouldNotReachHere(); + } + __ mfcr(R0); // set bit 32..33 as follows: <: 0b10, =: 0b00, >: 0b01 + __ srwi(Rdst, R0, 30); + __ srawi(R0, R0, 31); + __ orr(Rdst, R0, Rdst); // set result as follows: <: -1, =: 0, >: 1 + __ bind(done); +} + + +inline void load_to_reg(LIR_Assembler *lasm, LIR_Opr src, LIR_Opr dst) { + if (src->is_constant()) { + lasm->const2reg(src, dst, lir_patch_none, NULL); + } else if (src->is_register()) { + lasm->reg2reg(src, dst); + } else if (src->is_stack()) { + lasm->stack2reg(src, dst, dst->type()); + } else { + ShouldNotReachHere(); + } +} + + +void LIR_Assembler::cmove(LIR_Condition condition, LIR_Opr opr1, LIR_Opr opr2, LIR_Opr result, BasicType type) { + if (opr1->is_equal(opr2) || opr1->is_same_register(opr2)) { + load_to_reg(this, opr1, result); // Condition doesn't matter. + return; + } + + bool positive = false; + Assembler::Condition cond = Assembler::equal; + switch (condition) { + case lir_cond_equal: positive = true ; cond = Assembler::equal ; break; + case lir_cond_notEqual: positive = false; cond = Assembler::equal ; break; + case lir_cond_less: positive = true ; cond = Assembler::less ; break; + case lir_cond_belowEqual: + case lir_cond_lessEqual: positive = false; cond = Assembler::greater; break; + case lir_cond_greater: positive = true ; cond = Assembler::greater; break; + case lir_cond_aboveEqual: + case lir_cond_greaterEqual: positive = false; cond = Assembler::less ; break; + default: ShouldNotReachHere(); + } + + // Try to use isel on >=Power7. + if (VM_Version::has_isel() && result->is_cpu_register()) { + bool o1_is_reg = opr1->is_cpu_register(), o2_is_reg = opr2->is_cpu_register(); + const Register result_reg = result->is_single_cpu() ? result->as_register() : result->as_register_lo(); + + // We can use result_reg to load one operand if not already in register. + Register first = o1_is_reg ? (opr1->is_single_cpu() ? opr1->as_register() : opr1->as_register_lo()) : result_reg, + second = o2_is_reg ? (opr2->is_single_cpu() ? opr2->as_register() : opr2->as_register_lo()) : result_reg; + + if (first != second) { + if (!o1_is_reg) { + load_to_reg(this, opr1, result); + } + + if (!o2_is_reg) { + load_to_reg(this, opr2, result); + } + + __ isel(result_reg, BOOL_RESULT, cond, !positive, first, second); + return; + } + } // isel + + load_to_reg(this, opr1, result); + + Label skip; + int bo = positive ? Assembler::bcondCRbiIs1 : Assembler::bcondCRbiIs0; + int bi = Assembler::bi0(BOOL_RESULT, cond); + __ bc(bo, bi, skip); + + load_to_reg(this, opr2, result); + __ bind(skip); +} + + +void LIR_Assembler::arith_op(LIR_Code code, LIR_Opr left, LIR_Opr right, LIR_Opr dest, + CodeEmitInfo* info, bool pop_fpu_stack) { + assert(info == NULL, "unused on this code path"); + assert(left->is_register(), "wrong items state"); + assert(dest->is_register(), "wrong items state"); + + if (right->is_register()) { + if (dest->is_float_kind()) { + + FloatRegister lreg, rreg, res; + if (right->is_single_fpu()) { + lreg = left->as_float_reg(); + rreg = right->as_float_reg(); + res = dest->as_float_reg(); + switch (code) { + case lir_add: __ fadds(res, lreg, rreg); break; + case lir_sub: __ fsubs(res, lreg, rreg); break; + case lir_mul: // fall through + case lir_mul_strictfp: __ fmuls(res, lreg, rreg); break; + case lir_div: // fall through + case lir_div_strictfp: __ fdivs(res, lreg, rreg); break; + default: ShouldNotReachHere(); + } + } else { + lreg = left->as_double_reg(); + rreg = right->as_double_reg(); + res = dest->as_double_reg(); + switch (code) { + case lir_add: __ fadd(res, lreg, rreg); break; + case lir_sub: __ fsub(res, lreg, rreg); break; + case lir_mul: // fall through + case lir_mul_strictfp: __ fmul(res, lreg, rreg); break; + case lir_div: // fall through + case lir_div_strictfp: __ fdiv(res, lreg, rreg); break; + default: ShouldNotReachHere(); + } + } + + } else if (dest->is_double_cpu()) { + + Register dst_lo = dest->as_register_lo(); + Register op1_lo = left->as_pointer_register(); + Register op2_lo = right->as_pointer_register(); + + switch (code) { + case lir_add: __ add(dst_lo, op1_lo, op2_lo); break; + case lir_sub: __ sub(dst_lo, op1_lo, op2_lo); break; + case lir_mul: __ mulld(dst_lo, op1_lo, op2_lo); break; + default: ShouldNotReachHere(); + } + } else { + assert (right->is_single_cpu(), "Just Checking"); + + Register lreg = left->as_register(); + Register res = dest->as_register(); + Register rreg = right->as_register(); + switch (code) { + case lir_add: __ add (res, lreg, rreg); break; + case lir_sub: __ sub (res, lreg, rreg); break; + case lir_mul: __ mullw(res, lreg, rreg); break; + default: ShouldNotReachHere(); + } + } + } else { + assert (right->is_constant(), "must be constant"); + + if (dest->is_single_cpu()) { + Register lreg = left->as_register(); + Register res = dest->as_register(); + int simm16 = right->as_constant_ptr()->as_jint(); + + switch (code) { + case lir_sub: assert(Assembler::is_simm16(-simm16), "cannot encode"); // see do_ArithmeticOp_Int + simm16 = -simm16; + case lir_add: if (res == lreg && simm16 == 0) break; + __ addi(res, lreg, simm16); break; + case lir_mul: if (res == lreg && simm16 == 1) break; + __ mulli(res, lreg, simm16); break; + default: ShouldNotReachHere(); + } + } else { + Register lreg = left->as_pointer_register(); + Register res = dest->as_register_lo(); + long con = right->as_constant_ptr()->as_jlong(); + assert(Assembler::is_simm16(con), "must be simm16"); + + switch (code) { + case lir_sub: assert(Assembler::is_simm16(-con), "cannot encode"); // see do_ArithmeticOp_Long + con = -con; + case lir_add: if (res == lreg && con == 0) break; + __ addi(res, lreg, (int)con); break; + case lir_mul: if (res == lreg && con == 1) break; + __ mulli(res, lreg, (int)con); break; + default: ShouldNotReachHere(); + } + } + } +} + + +void LIR_Assembler::fpop() { + Unimplemented(); + // do nothing +} + + +void LIR_Assembler::intrinsic_op(LIR_Code code, LIR_Opr value, LIR_Opr thread, LIR_Opr dest, LIR_Op* op) { + switch (code) { + case lir_sqrt: { + __ fsqrt(dest->as_double_reg(), value->as_double_reg()); + break; + } + case lir_abs: { + __ fabs(dest->as_double_reg(), value->as_double_reg()); + break; + } + default: { + ShouldNotReachHere(); + break; + } + } +} + + +void LIR_Assembler::logic_op(LIR_Code code, LIR_Opr left, LIR_Opr right, LIR_Opr dest) { + if (right->is_constant()) { // see do_LogicOp + long uimm; + Register d, l; + if (dest->is_single_cpu()) { + uimm = right->as_constant_ptr()->as_jint(); + d = dest->as_register(); + l = left->as_register(); + } else { + uimm = right->as_constant_ptr()->as_jlong(); + d = dest->as_register_lo(); + l = left->as_register_lo(); + } + long uimms = (unsigned long)uimm >> 16, + uimmss = (unsigned long)uimm >> 32; + + switch (code) { + case lir_logic_and: + if (uimmss != 0 || (uimms != 0 && (uimm & 0xFFFF) != 0) || is_power_of_2_long(uimm)) { + __ andi(d, l, uimm); // special cases + } else if (uimms != 0) { __ andis_(d, l, uimms); } + else { __ andi_(d, l, uimm); } + break; + + case lir_logic_or: + if (uimms != 0) { assert((uimm & 0xFFFF) == 0, "sanity"); __ oris(d, l, uimms); } + else { __ ori(d, l, uimm); } + break; + + case lir_logic_xor: + if (uimm == -1) { __ nand(d, l, l); } // special case + else if (uimms != 0) { assert((uimm & 0xFFFF) == 0, "sanity"); __ xoris(d, l, uimms); } + else { __ xori(d, l, uimm); } + break; + + default: ShouldNotReachHere(); + } + } else { + assert(right->is_register(), "right should be in register"); + + if (dest->is_single_cpu()) { + switch (code) { + case lir_logic_and: __ andr(dest->as_register(), left->as_register(), right->as_register()); break; + case lir_logic_or: __ orr (dest->as_register(), left->as_register(), right->as_register()); break; + case lir_logic_xor: __ xorr(dest->as_register(), left->as_register(), right->as_register()); break; + default: ShouldNotReachHere(); + } + } else { + Register l = (left->is_single_cpu() && left->is_oop_register()) ? left->as_register() : + left->as_register_lo(); + Register r = (right->is_single_cpu() && right->is_oop_register()) ? right->as_register() : + right->as_register_lo(); + + switch (code) { + case lir_logic_and: __ andr(dest->as_register_lo(), l, r); break; + case lir_logic_or: __ orr (dest->as_register_lo(), l, r); break; + case lir_logic_xor: __ xorr(dest->as_register_lo(), l, r); break; + default: ShouldNotReachHere(); + } + } + } +} + + +int LIR_Assembler::shift_amount(BasicType t) { + int elem_size = type2aelembytes(t); + switch (elem_size) { + case 1 : return 0; + case 2 : return 1; + case 4 : return 2; + case 8 : return 3; + } + ShouldNotReachHere(); + return -1; +} + + +void LIR_Assembler::throw_op(LIR_Opr exceptionPC, LIR_Opr exceptionOop, CodeEmitInfo* info) { + info->add_register_oop(exceptionOop); + + // Reuse the debug info from the safepoint poll for the throw op itself. + address pc_for_athrow = __ pc(); + int pc_for_athrow_offset = __ offset(); + //RelocationHolder rspec = internal_word_Relocation::spec(pc_for_athrow); + //__ relocate(rspec); + //__ load_const(exceptionPC->as_register(), pc_for_athrow, R0); + __ calculate_address_from_global_toc(exceptionPC->as_register(), pc_for_athrow, true, true, /*add_relocation*/ true); + add_call_info(pc_for_athrow_offset, info); // for exception handler + + address stub = Runtime1::entry_for(compilation()->has_fpu_code() ? Runtime1::handle_exception_id + : Runtime1::handle_exception_nofpu_id); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ bctr(); +} + + +void LIR_Assembler::unwind_op(LIR_Opr exceptionOop) { + // Note: Not used with EnableDebuggingOnDemand. + assert(exceptionOop->as_register() == R3, "should match"); + __ b(_unwind_handler_entry); +} + + +void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) { + Register src = op->src()->as_register(); + Register dst = op->dst()->as_register(); + Register src_pos = op->src_pos()->as_register(); + Register dst_pos = op->dst_pos()->as_register(); + Register length = op->length()->as_register(); + Register tmp = op->tmp()->as_register(); + Register tmp2 = R0; + + int flags = op->flags(); + ciArrayKlass* default_type = op->expected_type(); + BasicType basic_type = default_type != NULL ? default_type->element_type()->basic_type() : T_ILLEGAL; + if (basic_type == T_ARRAY) basic_type = T_OBJECT; + + // Set up the arraycopy stub information. + ArrayCopyStub* stub = op->stub(); + const int frame_resize = frame::abi_reg_args_size - sizeof(frame::jit_abi); // C calls need larger frame. + + // Always do stub if no type information is available. It's ok if + // the known type isn't loaded since the code sanity checks + // in debug mode and the type isn't required when we know the exact type + // also check that the type is an array type. + if (op->expected_type() == NULL) { + assert(src->is_nonvolatile() && src_pos->is_nonvolatile() && dst->is_nonvolatile() && dst_pos->is_nonvolatile() && + length->is_nonvolatile(), "must preserve"); + // 3 parms are int. Convert to long. + __ mr(R3_ARG1, src); + __ extsw(R4_ARG2, src_pos); + __ mr(R5_ARG3, dst); + __ extsw(R6_ARG4, dst_pos); + __ extsw(R7_ARG5, length); + address copyfunc_addr = StubRoutines::generic_arraycopy(); + + if (copyfunc_addr == NULL) { // Use C version if stub was not generated. + address entry = CAST_FROM_FN_PTR(address, Runtime1::arraycopy); + __ call_c_with_frame_resize(entry, frame_resize); + } else { +#ifndef PRODUCT + if (PrintC1Statistics) { + address counter = (address)&Runtime1::_generic_arraycopystub_cnt; + int simm16_offs = __ load_const_optimized(tmp, counter, tmp2, true); + __ lwz(R11_scratch1, simm16_offs, tmp); + __ addi(R11_scratch1, R11_scratch1, 1); + __ stw(R11_scratch1, simm16_offs, tmp); + } +#endif + __ call_c_with_frame_resize(copyfunc_addr, /*stub does not need resized frame*/ 0); + + __ nand(tmp, R3_RET, R3_RET); + __ subf(length, tmp, length); + __ add(src_pos, tmp, src_pos); + __ add(dst_pos, tmp, dst_pos); + } + + __ cmpwi(CCR0, R3_RET, 0); + __ bc_far_optimized(Assembler::bcondCRbiIs1, __ bi0(CCR0, Assembler::less), *stub->entry()); + __ bind(*stub->continuation()); + return; + } + + assert(default_type != NULL && default_type->is_array_klass(), "must be true at this point"); + Label cont, slow, copyfunc; + + bool simple_check_flag_set = flags & (LIR_OpArrayCopy::src_null_check | + LIR_OpArrayCopy::dst_null_check | + LIR_OpArrayCopy::src_pos_positive_check | + LIR_OpArrayCopy::dst_pos_positive_check | + LIR_OpArrayCopy::length_positive_check); + + // Use only one conditional branch for simple checks. + if (simple_check_flag_set) { + ConditionRegister combined_check = CCR1, tmp_check = CCR1; + + // Make sure src and dst are non-null. + if (flags & LIR_OpArrayCopy::src_null_check) { + __ cmpdi(combined_check, src, 0); + tmp_check = CCR0; + } + + if (flags & LIR_OpArrayCopy::dst_null_check) { + __ cmpdi(tmp_check, dst, 0); + if (tmp_check != combined_check) { + __ cror(combined_check, Assembler::equal, tmp_check, Assembler::equal); + } + tmp_check = CCR0; + } + + // Clear combined_check.eq if not already used. + if (tmp_check == combined_check) { + __ crandc(combined_check, Assembler::equal, combined_check, Assembler::equal); + tmp_check = CCR0; + } + + if (flags & LIR_OpArrayCopy::src_pos_positive_check) { + // Test src_pos register. + __ cmpwi(tmp_check, src_pos, 0); + __ cror(combined_check, Assembler::equal, tmp_check, Assembler::less); + } + + if (flags & LIR_OpArrayCopy::dst_pos_positive_check) { + // Test dst_pos register. + __ cmpwi(tmp_check, dst_pos, 0); + __ cror(combined_check, Assembler::equal, tmp_check, Assembler::less); + } + + if (flags & LIR_OpArrayCopy::length_positive_check) { + // Make sure length isn't negative. + __ cmpwi(tmp_check, length, 0); + __ cror(combined_check, Assembler::equal, tmp_check, Assembler::less); + } + + __ beq(combined_check, slow); + } + + // Higher 32bits must be null. + __ extsw(length, length); + + __ extsw(src_pos, src_pos); + if (flags & LIR_OpArrayCopy::src_range_check) { + __ lwz(tmp2, arrayOopDesc::length_offset_in_bytes(), src); + __ add(tmp, length, src_pos); + __ cmpld(CCR0, tmp2, tmp); + __ ble(CCR0, slow); + } + + __ extsw(dst_pos, dst_pos); + if (flags & LIR_OpArrayCopy::dst_range_check) { + __ lwz(tmp2, arrayOopDesc::length_offset_in_bytes(), dst); + __ add(tmp, length, dst_pos); + __ cmpld(CCR0, tmp2, tmp); + __ ble(CCR0, slow); + } + + int shift = shift_amount(basic_type); + + if (!(flags & LIR_OpArrayCopy::type_check)) { + __ b(cont); + } else { + // We don't know the array types are compatible. + if (basic_type != T_OBJECT) { + // Simple test for basic type arrays. + if (UseCompressedClassPointers) { + // We don't need decode because we just need to compare. + __ lwz(tmp, oopDesc::klass_offset_in_bytes(), src); + __ lwz(tmp2, oopDesc::klass_offset_in_bytes(), dst); + __ cmpw(CCR0, tmp, tmp2); + } else { + __ ld(tmp, oopDesc::klass_offset_in_bytes(), src); + __ ld(tmp2, oopDesc::klass_offset_in_bytes(), dst); + __ cmpd(CCR0, tmp, tmp2); + } + __ beq(CCR0, cont); + } else { + // For object arrays, if src is a sub class of dst then we can + // safely do the copy. + address copyfunc_addr = StubRoutines::checkcast_arraycopy(); + + const Register sub_klass = R5, super_klass = R4; // like CheckCast/InstanceOf + assert_different_registers(tmp, tmp2, sub_klass, super_klass); + + __ load_klass(sub_klass, src); + __ load_klass(super_klass, dst); + + __ check_klass_subtype_fast_path(sub_klass, super_klass, tmp, tmp2, + &cont, copyfunc_addr != NULL ? ©func : &slow, NULL); + + address slow_stc = Runtime1::entry_for(Runtime1::slow_subtype_check_id); + //__ load_const_optimized(tmp, slow_stc, tmp2); + __ calculate_address_from_global_toc(tmp, slow_stc, true, true, false); + __ mtctr(tmp); + __ bctrl(); // sets CR0 + __ beq(CCR0, cont); + + if (copyfunc_addr != NULL) { // Use stub if available. + __ bind(copyfunc); + // Src is not a sub class of dst so we have to do a + // per-element check. + int mask = LIR_OpArrayCopy::src_objarray|LIR_OpArrayCopy::dst_objarray; + if ((flags & mask) != mask) { + assert(flags & mask, "one of the two should be known to be an object array"); + + if (!(flags & LIR_OpArrayCopy::src_objarray)) { + __ load_klass(tmp, src); + } else if (!(flags & LIR_OpArrayCopy::dst_objarray)) { + __ load_klass(tmp, dst); + } + + __ lwz(tmp2, in_bytes(Klass::layout_helper_offset()), tmp); + + jint objArray_lh = Klass::array_layout_helper(T_OBJECT); + __ load_const_optimized(tmp, objArray_lh); + __ cmpw(CCR0, tmp, tmp2); + __ bne(CCR0, slow); + } + + Register src_ptr = R3_ARG1; + Register dst_ptr = R4_ARG2; + Register len = R5_ARG3; + Register chk_off = R6_ARG4; + Register super_k = R7_ARG5; + + __ addi(src_ptr, src, arrayOopDesc::base_offset_in_bytes(basic_type)); + __ addi(dst_ptr, dst, arrayOopDesc::base_offset_in_bytes(basic_type)); + if (shift == 0) { + __ add(src_ptr, src_pos, src_ptr); + __ add(dst_ptr, dst_pos, dst_ptr); + } else { + __ sldi(tmp, src_pos, shift); + __ sldi(tmp2, dst_pos, shift); + __ add(src_ptr, tmp, src_ptr); + __ add(dst_ptr, tmp2, dst_ptr); + } + + __ load_klass(tmp, dst); + __ mr(len, length); + + int ek_offset = in_bytes(ObjArrayKlass::element_klass_offset()); + __ ld(super_k, ek_offset, tmp); + + int sco_offset = in_bytes(Klass::super_check_offset_offset()); + __ lwz(chk_off, sco_offset, super_k); + + __ call_c_with_frame_resize(copyfunc_addr, /*stub does not need resized frame*/ 0); + +#ifndef PRODUCT + if (PrintC1Statistics) { + Label failed; + __ cmpwi(CCR0, R3_RET, 0); + __ bne(CCR0, failed); + address counter = (address)&Runtime1::_arraycopy_checkcast_cnt; + int simm16_offs = __ load_const_optimized(tmp, counter, tmp2, true); + __ lwz(R11_scratch1, simm16_offs, tmp); + __ addi(R11_scratch1, R11_scratch1, 1); + __ stw(R11_scratch1, simm16_offs, tmp); + __ bind(failed); + } +#endif + + __ nand(tmp, R3_RET, R3_RET); + __ cmpwi(CCR0, R3_RET, 0); + __ beq(CCR0, *stub->continuation()); + +#ifndef PRODUCT + if (PrintC1Statistics) { + address counter = (address)&Runtime1::_arraycopy_checkcast_attempt_cnt; + int simm16_offs = __ load_const_optimized(tmp, counter, tmp2, true); + __ lwz(R11_scratch1, simm16_offs, tmp); + __ addi(R11_scratch1, R11_scratch1, 1); + __ stw(R11_scratch1, simm16_offs, tmp); + } +#endif + + __ subf(length, tmp, length); + __ add(src_pos, tmp, src_pos); + __ add(dst_pos, tmp, dst_pos); + } + } + } + __ bind(slow); + __ b(*stub->entry()); + __ bind(cont); + +#ifdef ASSERT + if (basic_type != T_OBJECT || !(flags & LIR_OpArrayCopy::type_check)) { + // Sanity check the known type with the incoming class. For the + // primitive case the types must match exactly with src.klass and + // dst.klass each exactly matching the default type. For the + // object array case, if no type check is needed then either the + // dst type is exactly the expected type and the src type is a + // subtype which we can't check or src is the same array as dst + // but not necessarily exactly of type default_type. + Label known_ok, halt; + metadata2reg(op->expected_type()->constant_encoding(), tmp); + if (UseCompressedClassPointers) { + // Tmp holds the default type. It currently comes uncompressed after the + // load of a constant, so encode it. + __ encode_klass_not_null(tmp); + // Load the raw value of the dst klass, since we will be comparing + // uncompressed values directly. + __ lwz(tmp2, oopDesc::klass_offset_in_bytes(), dst); + __ cmpw(CCR0, tmp, tmp2); + if (basic_type != T_OBJECT) { + __ bne(CCR0, halt); + // Load the raw value of the src klass. + __ lwz(tmp2, oopDesc::klass_offset_in_bytes(), src); + __ cmpw(CCR0, tmp, tmp2); + __ beq(CCR0, known_ok); + } else { + __ beq(CCR0, known_ok); + __ cmpw(CCR0, src, dst); + __ beq(CCR0, known_ok); + } + } else { + __ ld(tmp2, oopDesc::klass_offset_in_bytes(), dst); + __ cmpd(CCR0, tmp, tmp2); + if (basic_type != T_OBJECT) { + __ bne(CCR0, halt); + // Load the raw value of the src klass. + __ ld(tmp2, oopDesc::klass_offset_in_bytes(), src); + __ cmpd(CCR0, tmp, tmp2); + __ beq(CCR0, known_ok); + } else { + __ beq(CCR0, known_ok); + __ cmpd(CCR0, src, dst); + __ beq(CCR0, known_ok); + } + } + __ bind(halt); + __ stop("incorrect type information in arraycopy"); + __ bind(known_ok); + } +#endif + +#ifndef PRODUCT + if (PrintC1Statistics) { + address counter = Runtime1::arraycopy_count_address(basic_type); + int simm16_offs = __ load_const_optimized(tmp, counter, tmp2, true); + __ lwz(R11_scratch1, simm16_offs, tmp); + __ addi(R11_scratch1, R11_scratch1, 1); + __ stw(R11_scratch1, simm16_offs, tmp); + } +#endif + + Register src_ptr = R3_ARG1; + Register dst_ptr = R4_ARG2; + Register len = R5_ARG3; + + __ addi(src_ptr, src, arrayOopDesc::base_offset_in_bytes(basic_type)); + __ addi(dst_ptr, dst, arrayOopDesc::base_offset_in_bytes(basic_type)); + if (shift == 0) { + __ add(src_ptr, src_pos, src_ptr); + __ add(dst_ptr, dst_pos, dst_ptr); + } else { + __ sldi(tmp, src_pos, shift); + __ sldi(tmp2, dst_pos, shift); + __ add(src_ptr, tmp, src_ptr); + __ add(dst_ptr, tmp2, dst_ptr); + } + + bool disjoint = (flags & LIR_OpArrayCopy::overlapping) == 0; + bool aligned = (flags & LIR_OpArrayCopy::unaligned) == 0; + const char *name; + address entry = StubRoutines::select_arraycopy_function(basic_type, aligned, disjoint, name, false); + + // Arraycopy stubs takes a length in number of elements, so don't scale it. + __ mr(len, length); + __ call_c_with_frame_resize(entry, /*stub does not need resized frame*/ 0); + + __ bind(*stub->continuation()); +} + + +void LIR_Assembler::shift_op(LIR_Code code, LIR_Opr left, LIR_Opr count, LIR_Opr dest, LIR_Opr tmp) { + if (dest->is_single_cpu()) { + __ rldicl(tmp->as_register(), count->as_register(), 0, 64-5); +#ifdef _LP64 + if (left->type() == T_OBJECT) { + switch (code) { + case lir_shl: __ sld(dest->as_register(), left->as_register(), tmp->as_register()); break; + case lir_shr: __ srad(dest->as_register(), left->as_register(), tmp->as_register()); break; + case lir_ushr: __ srd(dest->as_register(), left->as_register(), tmp->as_register()); break; + default: ShouldNotReachHere(); + } + } else +#endif + switch (code) { + case lir_shl: __ slw(dest->as_register(), left->as_register(), tmp->as_register()); break; + case lir_shr: __ sraw(dest->as_register(), left->as_register(), tmp->as_register()); break; + case lir_ushr: __ srw(dest->as_register(), left->as_register(), tmp->as_register()); break; + default: ShouldNotReachHere(); + } + } else { + __ rldicl(tmp->as_register(), count->as_register(), 0, 64-6); + switch (code) { + case lir_shl: __ sld(dest->as_register_lo(), left->as_register_lo(), tmp->as_register()); break; + case lir_shr: __ srad(dest->as_register_lo(), left->as_register_lo(), tmp->as_register()); break; + case lir_ushr: __ srd(dest->as_register_lo(), left->as_register_lo(), tmp->as_register()); break; + default: ShouldNotReachHere(); + } + } +} + + +void LIR_Assembler::shift_op(LIR_Code code, LIR_Opr left, jint count, LIR_Opr dest) { +#ifdef _LP64 + if (left->type() == T_OBJECT) { + count = count & 63; // Shouldn't shift by more than sizeof(intptr_t). + if (count == 0) { __ mr_if_needed(dest->as_register_lo(), left->as_register()); } + else { + switch (code) { + case lir_shl: __ sldi(dest->as_register_lo(), left->as_register(), count); break; + case lir_shr: __ sradi(dest->as_register_lo(), left->as_register(), count); break; + case lir_ushr: __ srdi(dest->as_register_lo(), left->as_register(), count); break; + default: ShouldNotReachHere(); + } + } + return; + } +#endif + + if (dest->is_single_cpu()) { + count = count & 0x1F; // Java spec + if (count == 0) { __ mr_if_needed(dest->as_register(), left->as_register()); } + else { + switch (code) { + case lir_shl: __ slwi(dest->as_register(), left->as_register(), count); break; + case lir_shr: __ srawi(dest->as_register(), left->as_register(), count); break; + case lir_ushr: __ srwi(dest->as_register(), left->as_register(), count); break; + default: ShouldNotReachHere(); + } + } + } else if (dest->is_double_cpu()) { + count = count & 63; // Java spec + if (count == 0) { __ mr_if_needed(dest->as_pointer_register(), left->as_pointer_register()); } + else { + switch (code) { + case lir_shl: __ sldi(dest->as_pointer_register(), left->as_pointer_register(), count); break; + case lir_shr: __ sradi(dest->as_pointer_register(), left->as_pointer_register(), count); break; + case lir_ushr: __ srdi(dest->as_pointer_register(), left->as_pointer_register(), count); break; + default: ShouldNotReachHere(); + } + } + } else { + ShouldNotReachHere(); + } +} + + +void LIR_Assembler::emit_alloc_obj(LIR_OpAllocObj* op) { + if (op->init_check()) { + if (!os::zero_page_read_protected() || !ImplicitNullChecks) { + explicit_null_check(op->klass()->as_register(), op->stub()->info()); + } else { + add_debug_info_for_null_check_here(op->stub()->info()); + } + __ lbz(op->tmp1()->as_register(), + in_bytes(InstanceKlass::init_state_offset()), op->klass()->as_register()); + __ cmpwi(CCR0, op->tmp1()->as_register(), InstanceKlass::fully_initialized); + __ bc_far_optimized(Assembler::bcondCRbiIs0, __ bi0(CCR0, Assembler::equal), *op->stub()->entry()); + } + __ allocate_object(op->obj()->as_register(), + op->tmp1()->as_register(), + op->tmp2()->as_register(), + op->tmp3()->as_register(), + op->header_size(), + op->object_size(), + op->klass()->as_register(), + *op->stub()->entry()); + + __ bind(*op->stub()->continuation()); + __ verify_oop(op->obj()->as_register()); +} + + +void LIR_Assembler::emit_alloc_array(LIR_OpAllocArray* op) { + LP64_ONLY( __ extsw(op->len()->as_register(), op->len()->as_register()); ) + if (UseSlowPath || + (!UseFastNewObjectArray && (op->type() == T_OBJECT || op->type() == T_ARRAY)) || + (!UseFastNewTypeArray && (op->type() != T_OBJECT && op->type() != T_ARRAY))) { + __ b(*op->stub()->entry()); + } else { + __ allocate_array(op->obj()->as_register(), + op->len()->as_register(), + op->tmp1()->as_register(), + op->tmp2()->as_register(), + op->tmp3()->as_register(), + arrayOopDesc::header_size(op->type()), + type2aelembytes(op->type()), + op->klass()->as_register(), + *op->stub()->entry()); + } + __ bind(*op->stub()->continuation()); +} + + +void LIR_Assembler::type_profile_helper(Register mdo, int mdo_offset_bias, + ciMethodData *md, ciProfileData *data, + Register recv, Register tmp1, Label* update_done) { + uint i; + for (i = 0; i < VirtualCallData::row_limit(); i++) { + Label next_test; + // See if the receiver is receiver[n]. + __ ld(tmp1, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_offset(i)) - mdo_offset_bias, mdo); + __ verify_klass_ptr(tmp1); + __ cmpd(CCR0, recv, tmp1); + __ bne(CCR0, next_test); + + __ ld(tmp1, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + __ b(*update_done); + + __ bind(next_test); + } + + // Didn't find receiver; find next empty slot and fill it in. + for (i = 0; i < VirtualCallData::row_limit(); i++) { + Label next_test; + __ ld(tmp1, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_offset(i)) - mdo_offset_bias, mdo); + __ cmpdi(CCR0, tmp1, 0); + __ bne(CCR0, next_test); + __ li(tmp1, DataLayout::counter_increment); + __ std(recv, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_offset(i)) - mdo_offset_bias, mdo); + __ std(tmp1, md->byte_offset_of_slot(data, ReceiverTypeData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + __ b(*update_done); + + __ bind(next_test); + } +} + + +void LIR_Assembler::setup_md_access(ciMethod* method, int bci, + ciMethodData*& md, ciProfileData*& data, int& mdo_offset_bias) { + md = method->method_data_or_null(); + assert(md != NULL, "Sanity"); + data = md->bci_to_data(bci); + assert(data != NULL, "need data for checkcast"); + assert(data->is_ReceiverTypeData(), "need ReceiverTypeData for type check"); + if (!Assembler::is_simm16(md->byte_offset_of_slot(data, DataLayout::header_offset()) + data->size_in_bytes())) { + // The offset is large so bias the mdo by the base of the slot so + // that the ld can use simm16s to reference the slots of the data. + mdo_offset_bias = md->byte_offset_of_slot(data, DataLayout::header_offset()); + } +} + + +void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, Label* failure, Label* obj_is_null) { + Register obj = op->object()->as_register(); + Register k_RInfo = op->tmp1()->as_register(); + Register klass_RInfo = op->tmp2()->as_register(); + Register Rtmp1 = op->tmp3()->as_register(); + Register dst = op->result_opr()->as_register(); + ciKlass* k = op->klass(); + bool should_profile = op->should_profile(); + bool move_obj_to_dst = (op->code() == lir_checkcast); + // Attention: do_temp(opTypeCheck->_object) is not used, i.e. obj may be same as one of the temps. + bool reg_conflict = (obj == k_RInfo || obj == klass_RInfo || obj == Rtmp1); + bool restore_obj = move_obj_to_dst && reg_conflict; + + __ cmpdi(CCR0, obj, 0); + if (move_obj_to_dst || reg_conflict) { + __ mr_if_needed(dst, obj); + if (reg_conflict) { obj = dst; } + } + + ciMethodData* md; + ciProfileData* data; + int mdo_offset_bias = 0; + if (should_profile) { + ciMethod* method = op->profiled_method(); + assert(method != NULL, "Should have method"); + setup_md_access(method, op->profiled_bci(), md, data, mdo_offset_bias); + + Register mdo = k_RInfo; + Register data_val = Rtmp1; + Label not_null; + __ bne(CCR0, not_null); + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + __ lbz(data_val, md->byte_offset_of_slot(data, DataLayout::flags_offset()) - mdo_offset_bias, mdo); + __ ori(data_val, data_val, BitData::null_seen_byte_constant()); + __ stb(data_val, md->byte_offset_of_slot(data, DataLayout::flags_offset()) - mdo_offset_bias, mdo); + __ b(*obj_is_null); + __ bind(not_null); + } else { + __ beq(CCR0, *obj_is_null); + } + + // get object class + __ load_klass(klass_RInfo, obj); + + if (k->is_loaded()) { + metadata2reg(k->constant_encoding(), k_RInfo); + } else { + klass2reg_with_patching(k_RInfo, op->info_for_patch()); + } + + Label profile_cast_failure, failure_restore_obj, profile_cast_success; + Label *failure_target = should_profile ? &profile_cast_failure : failure; + Label *success_target = should_profile ? &profile_cast_success : success; + + if (op->fast_check()) { + assert_different_registers(klass_RInfo, k_RInfo); + __ cmpd(CCR0, k_RInfo, klass_RInfo); + if (should_profile) { + __ bne(CCR0, *failure_target); + // Fall through to success case. + } else { + __ beq(CCR0, *success); + // Fall through to failure case. + } + } else { + bool need_slow_path = true; + if (k->is_loaded()) { + if ((int) k->super_check_offset() != in_bytes(Klass::secondary_super_cache_offset())) { + need_slow_path = false; + } + // Perform the fast part of the checking logic. + __ check_klass_subtype_fast_path(klass_RInfo, k_RInfo, Rtmp1, R0, (need_slow_path ? success_target : NULL), + failure_target, NULL, RegisterOrConstant(k->super_check_offset())); + } else { + // Perform the fast part of the checking logic. + __ check_klass_subtype_fast_path(klass_RInfo, k_RInfo, Rtmp1, R0, success_target, failure_target); + } + if (!need_slow_path) { + if (!should_profile) { __ b(*success); } + } else { + // Call out-of-line instance of __ check_klass_subtype_slow_path(...): + address entry = Runtime1::entry_for(Runtime1::slow_subtype_check_id); + //__ load_const_optimized(Rtmp1, entry, R0); + __ calculate_address_from_global_toc(Rtmp1, entry, true, true, false); + __ mtctr(Rtmp1); + __ bctrl(); // sets CR0 + if (should_profile) { + __ bne(CCR0, *failure_target); + // Fall through to success case. + } else { + __ beq(CCR0, *success); + // Fall through to failure case. + } + } + } + + if (should_profile) { + Register mdo = k_RInfo, recv = klass_RInfo; + assert_different_registers(mdo, recv, Rtmp1); + __ bind(profile_cast_success); + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + type_profile_helper(mdo, mdo_offset_bias, md, data, recv, Rtmp1, success); + __ b(*success); + + // Cast failure case. + __ bind(profile_cast_failure); + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + __ ld(Rtmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + __ addi(Rtmp1, Rtmp1, -DataLayout::counter_increment); + __ std(Rtmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + } + + __ bind(*failure); + + if (restore_obj) { + __ mr(op->object()->as_register(), dst); + // Fall through to failure case. + } +} + + +void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) { + LIR_Code code = op->code(); + if (code == lir_store_check) { + Register value = op->object()->as_register(); + Register array = op->array()->as_register(); + Register k_RInfo = op->tmp1()->as_register(); + Register klass_RInfo = op->tmp2()->as_register(); + Register Rtmp1 = op->tmp3()->as_register(); + bool should_profile = op->should_profile(); + + __ verify_oop(value); + CodeStub* stub = op->stub(); + // Check if it needs to be profiled. + ciMethodData* md; + ciProfileData* data; + int mdo_offset_bias = 0; + if (should_profile) { + ciMethod* method = op->profiled_method(); + assert(method != NULL, "Should have method"); + setup_md_access(method, op->profiled_bci(), md, data, mdo_offset_bias); + } + Label profile_cast_success, failure, done; + Label *success_target = should_profile ? &profile_cast_success : &done; + + __ cmpdi(CCR0, value, 0); + if (should_profile) { + Label not_null; + __ bne(CCR0, not_null); + Register mdo = k_RInfo; + Register data_val = Rtmp1; + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + __ lbz(data_val, md->byte_offset_of_slot(data, DataLayout::flags_offset()) - mdo_offset_bias, mdo); + __ ori(data_val, data_val, BitData::null_seen_byte_constant()); + __ stb(data_val, md->byte_offset_of_slot(data, DataLayout::flags_offset()) - mdo_offset_bias, mdo); + __ b(done); + __ bind(not_null); + } else { + __ beq(CCR0, done); + } + if (!os::zero_page_read_protected() || !ImplicitNullChecks) { + explicit_null_check(array, op->info_for_exception()); + } else { + add_debug_info_for_null_check_here(op->info_for_exception()); + } + __ load_klass(k_RInfo, array); + __ load_klass(klass_RInfo, value); + + // Get instance klass. + __ ld(k_RInfo, in_bytes(ObjArrayKlass::element_klass_offset()), k_RInfo); + // Perform the fast part of the checking logic. + __ check_klass_subtype_fast_path(klass_RInfo, k_RInfo, Rtmp1, R0, success_target, &failure, NULL); + + // Call out-of-line instance of __ check_klass_subtype_slow_path(...): + const address slow_path = Runtime1::entry_for(Runtime1::slow_subtype_check_id); + //__ load_const_optimized(R0, slow_path); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(slow_path)); + __ mtctr(R0); + __ bctrl(); // sets CR0 + if (!should_profile) { + __ beq(CCR0, done); + __ bind(failure); + } else { + __ bne(CCR0, failure); + // Fall through to the success case. + + Register mdo = klass_RInfo, recv = k_RInfo, tmp1 = Rtmp1; + assert_different_registers(value, mdo, recv, tmp1); + __ bind(profile_cast_success); + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + __ load_klass(recv, value); + type_profile_helper(mdo, mdo_offset_bias, md, data, recv, tmp1, &done); + __ b(done); + + // Cast failure case. + __ bind(failure); + metadata2reg(md->constant_encoding(), mdo); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + Address data_addr(mdo, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias); + __ ld(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, -DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + } + __ b(*stub->entry()); + __ bind(done); + + } else if (code == lir_checkcast) { + Label success, failure; + emit_typecheck_helper(op, &success, /*fallthru*/&failure, &success); // Moves obj to dst. + __ b(*op->stub()->entry()); + __ align(32, 12); + __ bind(success); + } else if (code == lir_instanceof) { + Register dst = op->result_opr()->as_register(); + Label success, failure, done; + emit_typecheck_helper(op, &success, /*fallthru*/&failure, &failure); + __ li(dst, 0); + __ b(done); + __ align(32, 12); + __ bind(success); + __ li(dst, 1); + __ bind(done); + } else { + ShouldNotReachHere(); + } +} + + +void LIR_Assembler::emit_compare_and_swap(LIR_OpCompareAndSwap* op) { + Register addr = op->addr()->as_pointer_register(); + Register cmp_value = noreg, new_value = noreg; + bool is_64bit = false; + + if (op->code() == lir_cas_long) { + cmp_value = op->cmp_value()->as_register_lo(); + new_value = op->new_value()->as_register_lo(); + is_64bit = true; + } else if (op->code() == lir_cas_int || op->code() == lir_cas_obj) { + cmp_value = op->cmp_value()->as_register(); + new_value = op->new_value()->as_register(); + if (op->code() == lir_cas_obj) { + if (UseCompressedOops) { + Register t1 = op->tmp1()->as_register(); + Register t2 = op->tmp2()->as_register(); + cmp_value = __ encode_heap_oop(t1, cmp_value); + new_value = __ encode_heap_oop(t2, new_value); + } else { + is_64bit = true; + } + } + } else { + Unimplemented(); + } + + if (is_64bit) { + __ cmpxchgd(BOOL_RESULT, /*current_value=*/R0, cmp_value, new_value, addr, + MacroAssembler::MemBarFenceAfter, + MacroAssembler::cmpxchgx_hint_atomic_update(), + noreg, NULL, /*check without ldarx first*/true); + } else { + __ cmpxchgw(BOOL_RESULT, /*current_value=*/R0, cmp_value, new_value, addr, + MacroAssembler::MemBarFenceAfter, + MacroAssembler::cmpxchgx_hint_atomic_update(), + noreg, /*check without ldarx first*/true); + } +} + + +void LIR_Assembler::set_24bit_FPU() { + Unimplemented(); +} + +void LIR_Assembler::reset_FPU() { + Unimplemented(); +} + + +void LIR_Assembler::breakpoint() { + __ illtrap(); +} + + +void LIR_Assembler::push(LIR_Opr opr) { + Unimplemented(); +} + +void LIR_Assembler::pop(LIR_Opr opr) { + Unimplemented(); +} + + +void LIR_Assembler::monitor_address(int monitor_no, LIR_Opr dst_opr) { + Address mon_addr = frame_map()->address_for_monitor_lock(monitor_no); + Register dst = dst_opr->as_register(); + Register reg = mon_addr.base(); + int offset = mon_addr.disp(); + // Compute pointer to BasicLock. + __ add_const_optimized(dst, reg, offset); +} + + +void LIR_Assembler::emit_lock(LIR_OpLock* op) { + Register obj = op->obj_opr()->as_register(); + Register hdr = op->hdr_opr()->as_register(); + Register lock = op->lock_opr()->as_register(); + + // Obj may not be an oop. + if (op->code() == lir_lock) { + MonitorEnterStub* stub = (MonitorEnterStub*)op->stub(); + if (UseFastLocking) { + assert(BasicLock::displaced_header_offset_in_bytes() == 0, "lock_reg must point to the displaced header"); + // Add debug info for NullPointerException only if one is possible. + if (op->info() != NULL) { + if (!os::zero_page_read_protected() || !ImplicitNullChecks) { + explicit_null_check(obj, op->info()); + } else { + add_debug_info_for_null_check_here(op->info()); + } + } + __ lock_object(hdr, obj, lock, op->scratch_opr()->as_register(), *op->stub()->entry()); + } else { + // always do slow locking + // note: The slow locking code could be inlined here, however if we use + // slow locking, speed doesn't matter anyway and this solution is + // simpler and requires less duplicated code - additionally, the + // slow locking code is the same in either case which simplifies + // debugging. + __ b(*op->stub()->entry()); + } + } else { + assert (op->code() == lir_unlock, "Invalid code, expected lir_unlock"); + if (UseFastLocking) { + assert(BasicLock::displaced_header_offset_in_bytes() == 0, "lock_reg must point to the displaced header"); + __ unlock_object(hdr, obj, lock, *op->stub()->entry()); + } else { + // always do slow unlocking + // note: The slow unlocking code could be inlined here, however if we use + // slow unlocking, speed doesn't matter anyway and this solution is + // simpler and requires less duplicated code - additionally, the + // slow unlocking code is the same in either case which simplifies + // debugging. + __ b(*op->stub()->entry()); + } + } + __ bind(*op->stub()->continuation()); +} + + +void LIR_Assembler::emit_profile_call(LIR_OpProfileCall* op) { + ciMethod* method = op->profiled_method(); + int bci = op->profiled_bci(); + ciMethod* callee = op->profiled_callee(); + + // Update counter for all call types. + ciMethodData* md = method->method_data_or_null(); + assert(md != NULL, "Sanity"); + ciProfileData* data = md->bci_to_data(bci); + assert(data->is_CounterData(), "need CounterData for calls"); + assert(op->mdo()->is_single_cpu(), "mdo must be allocated"); + Register mdo = op->mdo()->as_register(); +#ifdef _LP64 + assert(op->tmp1()->is_double_cpu(), "tmp1 must be allocated"); + Register tmp1 = op->tmp1()->as_register_lo(); +#else + assert(op->tmp1()->is_single_cpu(), "tmp1 must be allocated"); + Register tmp1 = op->tmp1()->as_register(); +#endif + metadata2reg(md->constant_encoding(), mdo); + int mdo_offset_bias = 0; + if (!Assembler::is_simm16(md->byte_offset_of_slot(data, CounterData::count_offset()) + + data->size_in_bytes())) { + // The offset is large so bias the mdo by the base of the slot so + // that the ld can use simm16s to reference the slots of the data. + mdo_offset_bias = md->byte_offset_of_slot(data, CounterData::count_offset()); + __ add_const_optimized(mdo, mdo, mdo_offset_bias, R0); + } + + Bytecodes::Code bc = method->java_code_at_bci(bci); + const bool callee_is_static = callee->is_loaded() && callee->is_static(); + // Perform additional virtual call profiling for invokevirtual and + // invokeinterface bytecodes. + if ((bc == Bytecodes::_invokevirtual || bc == Bytecodes::_invokeinterface) && + !callee_is_static && // Required for optimized MH invokes. + C1ProfileVirtualCalls) { + assert(op->recv()->is_single_cpu(), "recv must be allocated"); + Register recv = op->recv()->as_register(); + assert_different_registers(mdo, tmp1, recv); + assert(data->is_VirtualCallData(), "need VirtualCallData for virtual calls"); + ciKlass* known_klass = op->known_holder(); + if (C1OptimizeVirtualCallProfiling && known_klass != NULL) { + // We know the type that will be seen at this call site; we can + // statically update the MethodData* rather than needing to do + // dynamic tests on the receiver type. + + // NOTE: we should probably put a lock around this search to + // avoid collisions by concurrent compilations. + ciVirtualCallData* vc_data = (ciVirtualCallData*) data; + uint i; + for (i = 0; i < VirtualCallData::row_limit(); i++) { + ciKlass* receiver = vc_data->receiver(i); + if (known_klass->equals(receiver)) { + __ ld(tmp1, md->byte_offset_of_slot(data, VirtualCallData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, VirtualCallData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + return; + } + } + + // Receiver type not found in profile data; select an empty slot. + + // Note that this is less efficient than it should be because it + // always does a write to the receiver part of the + // VirtualCallData rather than just the first time. + for (i = 0; i < VirtualCallData::row_limit(); i++) { + ciKlass* receiver = vc_data->receiver(i); + if (receiver == NULL) { + metadata2reg(known_klass->constant_encoding(), tmp1); + __ std(tmp1, md->byte_offset_of_slot(data, VirtualCallData::receiver_offset(i)) - mdo_offset_bias, mdo); + + __ ld(tmp1, md->byte_offset_of_slot(data, VirtualCallData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, VirtualCallData::receiver_count_offset(i)) - mdo_offset_bias, mdo); + return; + } + } + } else { + __ load_klass(recv, recv); + Label update_done; + type_profile_helper(mdo, mdo_offset_bias, md, data, recv, tmp1, &update_done); + // Receiver did not match any saved receiver and there is no empty row for it. + // Increment total counter to indicate polymorphic case. + __ ld(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + + __ bind(update_done); + } + } else { + // Static call + __ ld(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + __ addi(tmp1, tmp1, DataLayout::counter_increment); + __ std(tmp1, md->byte_offset_of_slot(data, CounterData::count_offset()) - mdo_offset_bias, mdo); + } +} + + +void LIR_Assembler::align_backward_branch_target() { + __ align(32, 12); // Insert up to 3 nops to align with 32 byte boundary. +} + + +void LIR_Assembler::emit_delay(LIR_OpDelay* op) { + Unimplemented(); +} + + +void LIR_Assembler::negate(LIR_Opr left, LIR_Opr dest) { + assert(left->is_register(), "can only handle registers"); + + if (left->is_single_cpu()) { + __ neg(dest->as_register(), left->as_register()); + } else if (left->is_single_fpu()) { + __ fneg(dest->as_float_reg(), left->as_float_reg()); + } else if (left->is_double_fpu()) { + __ fneg(dest->as_double_reg(), left->as_double_reg()); + } else { + assert (left->is_double_cpu(), "Must be a long"); + __ neg(dest->as_register_lo(), left->as_register_lo()); + } +} + + +void LIR_Assembler::fxch(int i) { + Unimplemented(); +} + +void LIR_Assembler::fld(int i) { + Unimplemented(); +} + +void LIR_Assembler::ffree(int i) { + Unimplemented(); +} + + +void LIR_Assembler::rt_call(LIR_Opr result, address dest, + const LIR_OprList* args, LIR_Opr tmp, CodeEmitInfo* info) { + // Stubs: Called via rt_call, but dest is a stub address (no function descriptor). + if (dest == Runtime1::entry_for(Runtime1::register_finalizer_id) || + dest == Runtime1::entry_for(Runtime1::new_multi_array_id )) { + //__ load_const_optimized(R0, dest); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(dest)); + __ mtctr(R0); + __ bctrl(); + assert(info != NULL, "sanity"); + add_call_info_here(info); + return; + } + + __ call_c_with_frame_resize(dest, /*no resizing*/ 0); + if (info != NULL) { + add_call_info_here(info); + } +} + + +void LIR_Assembler::volatile_move_op(LIR_Opr src, LIR_Opr dest, BasicType type, CodeEmitInfo* info) { + ShouldNotReachHere(); // Not needed on _LP64. +} + +void LIR_Assembler::membar() { + __ fence(); +} + +void LIR_Assembler::membar_acquire() { + __ acquire(); +} + +void LIR_Assembler::membar_release() { + __ release(); +} + +void LIR_Assembler::membar_loadload() { + __ membar(Assembler::LoadLoad); +} + +void LIR_Assembler::membar_storestore() { + __ membar(Assembler::StoreStore); +} + +void LIR_Assembler::membar_loadstore() { + __ membar(Assembler::LoadStore); +} + +void LIR_Assembler::membar_storeload() { + __ membar(Assembler::StoreLoad); +} + + +void LIR_Assembler::leal(LIR_Opr addr_opr, LIR_Opr dest) { + LIR_Address* addr = addr_opr->as_address_ptr(); + assert(addr->scale() == LIR_Address::times_1, "no scaling on this platform"); + if (addr->index()->is_illegal()) { + __ add_const_optimized(dest->as_pointer_register(), addr->base()->as_pointer_register(), addr->disp()); + } else { + assert(addr->disp() == 0, "can't have both: index and disp"); + __ add(dest->as_pointer_register(), addr->index()->as_pointer_register(), addr->base()->as_pointer_register()); + } +} + + +void LIR_Assembler::get_thread(LIR_Opr result_reg) { + ShouldNotReachHere(); +} + + +#ifdef ASSERT +// Emit run-time assertion. +void LIR_Assembler::emit_assert(LIR_OpAssert* op) { + Unimplemented(); +} +#endif + + +void LIR_Assembler::peephole(LIR_List* lir) { + // Optimize instruction pairs before emitting. + LIR_OpList* inst = lir->instructions_list(); + for (int i = 1; i < inst->length(); i++) { + LIR_Op* op = inst->at(i); + + // 2 register-register-moves + if (op->code() == lir_move) { + LIR_Opr in2 = ((LIR_Op1*)op)->in_opr(), + res2 = ((LIR_Op1*)op)->result_opr(); + if (in2->is_register() && res2->is_register()) { + LIR_Op* prev = inst->at(i - 1); + if (prev && prev->code() == lir_move) { + LIR_Opr in1 = ((LIR_Op1*)prev)->in_opr(), + res1 = ((LIR_Op1*)prev)->result_opr(); + if (in1->is_same_register(res2) && in2->is_same_register(res1)) { + inst->remove_at(i); + } + } + } + } + + } + return; +} + + +void LIR_Assembler::atomic_op(LIR_Code code, LIR_Opr src, LIR_Opr data, LIR_Opr dest, LIR_Opr tmp) { + const Register Rptr = src->as_pointer_register(), + Rtmp = tmp->as_register(); + Register Rco = noreg; + if (UseCompressedOops && data->is_oop()) { + Rco = __ encode_heap_oop(Rtmp, data->as_register()); + } + + Label Lretry; + __ bind(Lretry); + + if (data->type() == T_INT) { + const Register Rold = dest->as_register(), + Rsrc = data->as_register(); + assert_different_registers(Rptr, Rtmp, Rold, Rsrc); + __ lwarx(Rold, Rptr, MacroAssembler::cmpxchgx_hint_atomic_update()); + if (code == lir_xadd) { + __ add(Rtmp, Rsrc, Rold); + __ stwcx_(Rtmp, Rptr); + } else { + __ stwcx_(Rsrc, Rptr); + } + } else if (data->is_oop()) { + assert(code == lir_xchg, "xadd for oops"); + const Register Rold = dest->as_register(); + if (UseCompressedOops) { + assert_different_registers(Rptr, Rold, Rco); + __ lwarx(Rold, Rptr, MacroAssembler::cmpxchgx_hint_atomic_update()); + __ stwcx_(Rco, Rptr); + } else { + const Register Robj = data->as_register(); + assert_different_registers(Rptr, Rold, Robj); + __ ldarx(Rold, Rptr, MacroAssembler::cmpxchgx_hint_atomic_update()); + __ stdcx_(Robj, Rptr); + } + } else if (data->type() == T_LONG) { + const Register Rold = dest->as_register_lo(), + Rsrc = data->as_register_lo(); + assert_different_registers(Rptr, Rtmp, Rold, Rsrc); + __ ldarx(Rold, Rptr, MacroAssembler::cmpxchgx_hint_atomic_update()); + if (code == lir_xadd) { + __ add(Rtmp, Rsrc, Rold); + __ stdcx_(Rtmp, Rptr); + } else { + __ stdcx_(Rsrc, Rptr); + } + } else { + ShouldNotReachHere(); + } + + if (UseStaticBranchPredictionInCompareAndSwapPPC64) { + __ bne_predict_not_taken(CCR0, Lretry); + } else { + __ bne( CCR0, Lretry); + } + + if (UseCompressedOops && data->is_oop()) { + __ decode_heap_oop(dest->as_register()); + } +} + + +void LIR_Assembler::emit_profile_type(LIR_OpProfileType* op) { + Register obj = op->obj()->as_register(); + Register tmp = op->tmp()->as_pointer_register(); + LIR_Address* mdo_addr = op->mdp()->as_address_ptr(); + ciKlass* exact_klass = op->exact_klass(); + intptr_t current_klass = op->current_klass(); + bool not_null = op->not_null(); + bool no_conflict = op->no_conflict(); + + Label Lupdate, Ldo_update, Ldone; + + bool do_null = !not_null; + bool exact_klass_set = exact_klass != NULL && ciTypeEntries::valid_ciklass(current_klass) == exact_klass; + bool do_update = !TypeEntries::is_type_unknown(current_klass) && !exact_klass_set; + + assert(do_null || do_update, "why are we here?"); + assert(!TypeEntries::was_null_seen(current_klass) || do_update, "why are we here?"); + + __ verify_oop(obj); + + if (do_null) { + if (!TypeEntries::was_null_seen(current_klass)) { + __ cmpdi(CCR0, obj, 0); + __ bne(CCR0, Lupdate); + __ ld(R0, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + __ ori(R0, R0, TypeEntries::null_seen); + if (do_update) { + __ b(Ldo_update); + } else { + __ std(R0, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + } + } else { + if (do_update) { + __ cmpdi(CCR0, obj, 0); + __ beq(CCR0, Ldone); + } + } +#ifdef ASSERT + } else { + __ cmpdi(CCR0, obj, 0); + __ bne(CCR0, Lupdate); + __ stop("unexpect null obj", 0x9652); +#endif + } + + __ bind(Lupdate); + if (do_update) { + Label Lnext; + const Register klass = R29_TOC; // kill and reload + bool klass_reg_used = false; +#ifdef ASSERT + if (exact_klass != NULL) { + Label ok; + klass_reg_used = true; + __ load_klass(klass, obj); + metadata2reg(exact_klass->constant_encoding(), R0); + __ cmpd(CCR0, klass, R0); + __ beq(CCR0, ok); + __ stop("exact klass and actual klass differ", 0x8564); + __ bind(ok); + } +#endif + + if (!no_conflict) { + if (exact_klass == NULL || TypeEntries::is_type_none(current_klass)) { + klass_reg_used = true; + if (exact_klass != NULL) { + __ ld(tmp, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + metadata2reg(exact_klass->constant_encoding(), klass); + } else { + __ load_klass(klass, obj); + __ ld(tmp, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); // may kill obj + } + + // Like InterpreterMacroAssembler::profile_obj_type + __ clrrdi(R0, tmp, exact_log2(-TypeEntries::type_klass_mask)); + // Basically same as andi(R0, tmp, TypeEntries::type_klass_mask); + __ cmpd(CCR1, R0, klass); + // Klass seen before, nothing to do (regardless of unknown bit). + //beq(CCR1, do_nothing); + + __ andi_(R0, klass, TypeEntries::type_unknown); + // Already unknown. Nothing to do anymore. + //bne(CCR0, do_nothing); + __ crorc(CCR0, Assembler::equal, CCR1, Assembler::equal); // cr0 eq = cr1 eq or cr0 ne + __ beq(CCR0, Lnext); + + if (TypeEntries::is_type_none(current_klass)) { + __ clrrdi_(R0, tmp, exact_log2(-TypeEntries::type_mask)); + __ orr(R0, klass, tmp); // Combine klass and null_seen bit (only used if (tmp & type_mask)==0). + __ beq(CCR0, Ldo_update); // First time here. Set profile type. + } + + } else { + assert(ciTypeEntries::valid_ciklass(current_klass) != NULL && + ciTypeEntries::valid_ciklass(current_klass) != exact_klass, "conflict only"); + + __ ld(tmp, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + __ andi_(R0, tmp, TypeEntries::type_unknown); + // Already unknown. Nothing to do anymore. + __ bne(CCR0, Lnext); + } + + // Different than before. Cannot keep accurate profile. + __ ori(R0, tmp, TypeEntries::type_unknown); + } else { + // There's a single possible klass at this profile point + assert(exact_klass != NULL, "should be"); + __ ld(tmp, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + + if (TypeEntries::is_type_none(current_klass)) { + klass_reg_used = true; + metadata2reg(exact_klass->constant_encoding(), klass); + + __ clrrdi(R0, tmp, exact_log2(-TypeEntries::type_klass_mask)); + // Basically same as andi(R0, tmp, TypeEntries::type_klass_mask); + __ cmpd(CCR1, R0, klass); + // Klass seen before, nothing to do (regardless of unknown bit). + __ beq(CCR1, Lnext); +#ifdef ASSERT + { + Label ok; + __ clrrdi_(R0, tmp, exact_log2(-TypeEntries::type_mask)); + __ beq(CCR0, ok); // First time here. + + __ stop("unexpected profiling mismatch", 0x7865); + __ bind(ok); + } +#endif + // First time here. Set profile type. + __ orr(R0, klass, tmp); // Combine klass and null_seen bit (only used if (tmp & type_mask)==0). + } else { + assert(ciTypeEntries::valid_ciklass(current_klass) != NULL && + ciTypeEntries::valid_ciklass(current_klass) != exact_klass, "inconsistent"); + + // Already unknown. Nothing to do anymore. + __ andi_(R0, tmp, TypeEntries::type_unknown); + __ bne(CCR0, Lnext); + + // Different than before. Cannot keep accurate profile. + __ ori(R0, tmp, TypeEntries::type_unknown); + } + } + + __ bind(Ldo_update); + __ std(R0, index_or_disp(mdo_addr), mdo_addr->base()->as_pointer_register()); + + __ bind(Lnext); + if (klass_reg_used) { __ load_const_optimized(R29_TOC, MacroAssembler::global_toc(), R0); } // reinit + } + __ bind(Ldone); +} + + +void LIR_Assembler::emit_updatecrc32(LIR_OpUpdateCRC32* op) { + assert(op->crc()->is_single_cpu(), "crc must be register"); + assert(op->val()->is_single_cpu(), "byte value must be register"); + assert(op->result_opr()->is_single_cpu(), "result must be register"); + Register crc = op->crc()->as_register(); + Register val = op->val()->as_register(); + Register res = op->result_opr()->as_register(); + + assert_different_registers(val, crc, res); + + __ load_const_optimized(res, StubRoutines::crc_table_addr(), R0); + __ nand(crc, crc, crc); // ~crc + __ update_byte_crc32(crc, val, res); + __ nand(res, crc, crc); // ~crc +} + +#undef __ diff --git a/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.hpp new file mode 100644 index 00000000000..03d1faedbf9 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_LIRAssembler_ppc.hpp @@ -0,0 +1,69 @@ +/* + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_LIRASSEMBLER_PPC_HPP +#define CPU_PPC_VM_C1_LIRASSEMBLER_PPC_HPP + + private: + + ////////////////////////////////////////////////////////////////////////////// + // PPC64 load/store emission + // + // The PPC ld/st instructions cannot accomodate displacements > 16 bits long. + // The following "pseudo" instructions (load/store) make it easier to + // use the indexed addressing mode by allowing 32 bit displacements: + // + + void explicit_null_check(Register addr, CodeEmitInfo* info); + + int store(LIR_Opr from_reg, Register base, int offset, BasicType type, bool wide, bool unaligned); + int store(LIR_Opr from_reg, Register base, Register disp, BasicType type, bool wide); + + int load(Register base, int offset, LIR_Opr to_reg, BasicType type, bool wide, bool unaligned); + int load(Register base, Register disp, LIR_Opr to_reg, BasicType type, bool wide); + + int shift_amount(BasicType t); + + // Record the type of the receiver in ReceiverTypeData. + void type_profile_helper(Register mdo, int mdo_offset_bias, + ciMethodData *md, ciProfileData *data, + Register recv, Register tmp1, Label* update_done); + // Setup pointers to MDO, MDO slot, also compute offset bias to access the slot. + void setup_md_access(ciMethod* method, int bci, + ciMethodData*& md, ciProfileData*& data, int& mdo_offset_bias); + public: + static const ConditionRegister BOOL_RESULT; + + // Emit trampoline stub for call. Call bailout() if failed. Return true on success. + bool emit_trampoline_stub_for_call(address target, Register Rtoc = noreg); + +enum { + max_static_call_stub_size = 4 * BytesPerInstWord + MacroAssembler::b64_patchable_size, + call_stub_size = max_static_call_stub_size + MacroAssembler::trampoline_stub_size, // or smaller + exception_handler_size = MacroAssembler::b64_patchable_size, // or smaller + deopt_handler_size = MacroAssembler::bl64_patchable_size +}; + +#endif // CPU_PPC_VM_C1_LIRASSEMBLER_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_LIRGenerator_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_LIRGenerator_ppc.cpp new file mode 100644 index 00000000000..909d0136011 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_LIRGenerator_ppc.cpp @@ -0,0 +1,1429 @@ +/* + * Copyright (c) 2005, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_Compilation.hpp" +#include "c1/c1_FrameMap.hpp" +#include "c1/c1_Instruction.hpp" +#include "c1/c1_LIRAssembler.hpp" +#include "c1/c1_LIRGenerator.hpp" +#include "c1/c1_Runtime1.hpp" +#include "c1/c1_ValueStack.hpp" +#include "ci/ciArray.hpp" +#include "ci/ciObjArrayKlass.hpp" +#include "ci/ciTypeArrayKlass.hpp" +#include "runtime/sharedRuntime.hpp" +#include "runtime/stubRoutines.hpp" +#include "vmreg_ppc.inline.hpp" + +#ifdef ASSERT +#define __ gen()->lir(__FILE__, __LINE__)-> +#else +#define __ gen()->lir()-> +#endif + +void LIRItem::load_byte_item() { + // Byte loads use same registers as other loads. + load_item(); +} + + +void LIRItem::load_nonconstant() { + LIR_Opr r = value()->operand(); + if (_gen->can_inline_as_constant(value())) { + if (!r->is_constant()) { + r = LIR_OprFact::value_type(value()->type()); + } + _result = r; + } else { + load_item(); + } +} + + +inline void load_int_as_long(LIR_List *ll, LIRItem &li, LIR_Opr dst) { + LIR_Opr r = li.value()->operand(); + if (r->is_register()) { + LIR_Opr dst_l = FrameMap::as_long_opr(dst->as_register()); + ll->convert(Bytecodes::_i2l, li.result(), dst_l); // Convert. + } else { + // Constants or memory get loaded with sign extend on this platform. + ll->move(li.result(), dst); + } +} + + +//-------------------------------------------------------------- +// LIRGenerator +//-------------------------------------------------------------- + +LIR_Opr LIRGenerator::exceptionOopOpr() { return FrameMap::R3_oop_opr; } +LIR_Opr LIRGenerator::exceptionPcOpr() { return FrameMap::R4_opr; } +LIR_Opr LIRGenerator::syncLockOpr() { return FrameMap::R5_opr; } // Need temp effect for MonitorEnterStub. +LIR_Opr LIRGenerator::syncTempOpr() { return FrameMap::R4_oop_opr; } // Need temp effect for MonitorEnterStub. +LIR_Opr LIRGenerator::getThreadTemp() { return LIR_OprFact::illegalOpr; } // not needed + +LIR_Opr LIRGenerator::result_register_for(ValueType* type, bool callee) { + LIR_Opr opr; + switch (type->tag()) { + case intTag: opr = FrameMap::R3_opr; break; + case objectTag: opr = FrameMap::R3_oop_opr; break; + case longTag: opr = FrameMap::R3_long_opr; break; + case floatTag: opr = FrameMap::F1_opr; break; + case doubleTag: opr = FrameMap::F1_double_opr; break; + + case addressTag: + default: ShouldNotReachHere(); return LIR_OprFact::illegalOpr; + } + + assert(opr->type_field() == as_OprType(as_BasicType(type)), "type mismatch"); + return opr; +} + +LIR_Opr LIRGenerator::rlock_callee_saved(BasicType type) { + ShouldNotReachHere(); + return LIR_OprFact::illegalOpr; +} + + +LIR_Opr LIRGenerator::rlock_byte(BasicType type) { + return new_register(T_INT); +} + + +//--------- loading items into registers -------------------------------- + +// PPC cannot inline all constants. +bool LIRGenerator::can_store_as_constant(Value v, BasicType type) const { + if (v->type()->as_IntConstant() != NULL) { + return Assembler::is_simm16(v->type()->as_IntConstant()->value()); + } else if (v->type()->as_LongConstant() != NULL) { + return Assembler::is_simm16(v->type()->as_LongConstant()->value()); + } else if (v->type()->as_ObjectConstant() != NULL) { + return v->type()->as_ObjectConstant()->value()->is_null_object(); + } else { + return false; + } +} + + +// Only simm16 constants can be inlined. +bool LIRGenerator::can_inline_as_constant(Value i) const { + return can_store_as_constant(i, as_BasicType(i->type())); +} + + +bool LIRGenerator::can_inline_as_constant(LIR_Const* c) const { + if (c->type() == T_INT) { + return Assembler::is_simm16(c->as_jint()); + } + if (c->type() == T_LONG) { + return Assembler::is_simm16(c->as_jlong()); + } + if (c->type() == T_OBJECT) { + return c->as_jobject() == NULL; + } + return false; +} + + +LIR_Opr LIRGenerator::safepoint_poll_register() { + return new_register(T_INT); +} + + +LIR_Address* LIRGenerator::generate_address(LIR_Opr base, LIR_Opr index, + int shift, int disp, BasicType type) { + assert(base->is_register(), "must be"); + + // Accumulate fixed displacements. + if (index->is_constant()) { + disp += index->as_constant_ptr()->as_jint() << shift; + index = LIR_OprFact::illegalOpr; + } + + if (index->is_register()) { + // Apply the shift and accumulate the displacement. + if (shift > 0) { + LIR_Opr tmp = new_pointer_register(); + __ shift_left(index, shift, tmp); + index = tmp; + } + if (disp != 0) { + LIR_Opr tmp = new_pointer_register(); + if (Assembler::is_simm16(disp)) { + __ add(index, LIR_OprFact::intptrConst(disp), tmp); + index = tmp; + } else { + __ move(LIR_OprFact::intptrConst(disp), tmp); + __ add(tmp, index, tmp); + index = tmp; + } + disp = 0; + } + } else if (!Assembler::is_simm16(disp)) { + // Index is illegal so replace it with the displacement loaded into a register. + index = new_pointer_register(); + __ move(LIR_OprFact::intptrConst(disp), index); + disp = 0; + } + + // At this point we either have base + index or base + displacement. + if (disp == 0) { + return new LIR_Address(base, index, type); + } else { + assert(Assembler::is_simm16(disp), "must be"); + return new LIR_Address(base, disp, type); + } +} + + +LIR_Address* LIRGenerator::emit_array_address(LIR_Opr array_opr, LIR_Opr index_opr, + BasicType type, bool needs_card_mark) { + int elem_size = type2aelembytes(type); + int shift = exact_log2(elem_size); + + LIR_Opr base_opr; + int offset = arrayOopDesc::base_offset_in_bytes(type); + + if (index_opr->is_constant()) { + int i = index_opr->as_constant_ptr()->as_jint(); + int array_offset = i * elem_size; + if (Assembler::is_simm16(array_offset + offset)) { + base_opr = array_opr; + offset = array_offset + offset; + } else { + base_opr = new_pointer_register(); + if (Assembler::is_simm16(array_offset)) { + __ add(array_opr, LIR_OprFact::intptrConst(array_offset), base_opr); + } else { + __ move(LIR_OprFact::intptrConst(array_offset), base_opr); + __ add(base_opr, array_opr, base_opr); + } + } + } else { +#ifdef _LP64 + if (index_opr->type() == T_INT) { + LIR_Opr tmp = new_register(T_LONG); + __ convert(Bytecodes::_i2l, index_opr, tmp); + index_opr = tmp; + } +#endif + + base_opr = new_pointer_register(); + assert (index_opr->is_register(), "Must be register"); + if (shift > 0) { + __ shift_left(index_opr, shift, base_opr); + __ add(base_opr, array_opr, base_opr); + } else { + __ add(index_opr, array_opr, base_opr); + } + } + if (needs_card_mark) { + LIR_Opr ptr = new_pointer_register(); + __ add(base_opr, LIR_OprFact::intptrConst(offset), ptr); + return new LIR_Address(ptr, type); + } else { + return new LIR_Address(base_opr, offset, type); + } +} + + +LIR_Opr LIRGenerator::load_immediate(int x, BasicType type) { + LIR_Opr r = NULL; + if (type == T_LONG) { + r = LIR_OprFact::longConst(x); + } else if (type == T_INT) { + r = LIR_OprFact::intConst(x); + } else { + ShouldNotReachHere(); + } + if (!Assembler::is_simm16(x)) { + LIR_Opr tmp = new_register(type); + __ move(r, tmp); + return tmp; + } + return r; +} + + +void LIRGenerator::increment_counter(address counter, BasicType type, int step) { + LIR_Opr pointer = new_pointer_register(); + __ move(LIR_OprFact::intptrConst(counter), pointer); + LIR_Address* addr = new LIR_Address(pointer, type); + increment_counter(addr, step); +} + + +void LIRGenerator::increment_counter(LIR_Address* addr, int step) { + LIR_Opr temp = new_register(addr->type()); + __ move(addr, temp); + __ add(temp, load_immediate(step, addr->type()), temp); + __ move(temp, addr); +} + + +void LIRGenerator::cmp_mem_int(LIR_Condition condition, LIR_Opr base, int disp, int c, CodeEmitInfo* info) { + LIR_Opr tmp = FrameMap::R0_opr; + __ load(new LIR_Address(base, disp, T_INT), tmp, info); + __ cmp(condition, tmp, c); +} + + +void LIRGenerator::cmp_reg_mem(LIR_Condition condition, LIR_Opr reg, LIR_Opr base, + int disp, BasicType type, CodeEmitInfo* info) { + LIR_Opr tmp = FrameMap::R0_opr; + __ load(new LIR_Address(base, disp, type), tmp, info); + __ cmp(condition, reg, tmp); +} + + +void LIRGenerator::cmp_reg_mem(LIR_Condition condition, LIR_Opr reg, LIR_Opr base, + LIR_Opr disp, BasicType type, CodeEmitInfo* info) { + LIR_Opr tmp = FrameMap::R0_opr; + __ load(new LIR_Address(base, disp, type), tmp, info); + __ cmp(condition, reg, tmp); +} + + +bool LIRGenerator::strength_reduce_multiply(LIR_Opr left, int c, LIR_Opr result, LIR_Opr tmp) { + assert(left != result, "should be different registers"); + if (is_power_of_2(c + 1)) { + __ shift_left(left, log2_intptr(c + 1), result); + __ sub(result, left, result); + return true; + } else if (is_power_of_2(c - 1)) { + __ shift_left(left, log2_intptr(c - 1), result); + __ add(result, left, result); + return true; + } + return false; +} + + +void LIRGenerator::store_stack_parameter(LIR_Opr item, ByteSize offset_from_sp) { + BasicType t = item->type(); + LIR_Opr sp_opr = FrameMap::SP_opr; + if ((t == T_LONG || t == T_DOUBLE) && + ((in_bytes(offset_from_sp) - STACK_BIAS) % 8 != 0)) { + __ unaligned_move(item, new LIR_Address(sp_opr, in_bytes(offset_from_sp), t)); + } else { + __ move(item, new LIR_Address(sp_opr, in_bytes(offset_from_sp), t)); + } +} + + +//---------------------------------------------------------------------- +// visitor functions +//---------------------------------------------------------------------- + +void LIRGenerator::do_StoreIndexed(StoreIndexed* x) { + assert(x->is_pinned(),""); + bool needs_range_check = x->compute_needs_range_check(); + bool use_length = x->length() != NULL; + bool obj_store = x->elt_type() == T_ARRAY || x->elt_type() == T_OBJECT; + bool needs_store_check = obj_store && (x->value()->as_Constant() == NULL || + !get_jobject_constant(x->value())->is_null_object() || + x->should_profile()); + + LIRItem array(x->array(), this); + LIRItem index(x->index(), this); + LIRItem value(x->value(), this); + LIRItem length(this); + + array.load_item(); + index.load_nonconstant(); + + if (use_length && needs_range_check) { + length.set_instruction(x->length()); + length.load_item(); + } + if (needs_store_check) { + value.load_item(); + } else { + value.load_for_store(x->elt_type()); + } + + set_no_result(x); + + // The CodeEmitInfo must be duplicated for each different + // LIR-instruction because spilling can occur anywhere between two + // instructions and so the debug information must be different. + CodeEmitInfo* range_check_info = state_for(x); + CodeEmitInfo* null_check_info = NULL; + if (x->needs_null_check()) { + null_check_info = new CodeEmitInfo(range_check_info); + } + + // Emit array address setup early so it schedules better. + LIR_Address* array_addr = emit_array_address(array.result(), index.result(), x->elt_type(), obj_store); + + if (GenerateRangeChecks && needs_range_check) { + if (use_length) { + __ cmp(lir_cond_belowEqual, length.result(), index.result()); + __ branch(lir_cond_belowEqual, T_INT, new RangeCheckStub(range_check_info, index.result())); + } else { + array_range_check(array.result(), index.result(), null_check_info, range_check_info); + // Range_check also does the null check. + null_check_info = NULL; + } + } + + if (GenerateArrayStoreCheck && needs_store_check) { + // Following registers are used by slow_subtype_check: + LIR_Opr tmp1 = FrameMap::R4_opr; // super_klass + LIR_Opr tmp2 = FrameMap::R5_opr; // sub_klass + LIR_Opr tmp3 = FrameMap::R6_opr; // temp + + CodeEmitInfo* store_check_info = new CodeEmitInfo(range_check_info); + __ store_check(value.result(), array.result(), tmp1, tmp2, tmp3, + store_check_info, x->profiled_method(), x->profiled_bci()); + } + + if (obj_store) { + // Needs GC write barriers. + pre_barrier(LIR_OprFact::address(array_addr), LIR_OprFact::illegalOpr /* pre_val */, + true /* do_load */, false /* patch */, NULL); + } + __ move(value.result(), array_addr, null_check_info); + if (obj_store) { + // Precise card mark. + post_barrier(LIR_OprFact::address(array_addr), value.result()); + } +} + + +void LIRGenerator::do_MonitorEnter(MonitorEnter* x) { + assert(x->is_pinned(),""); + LIRItem obj(x->obj(), this); + obj.load_item(); + + set_no_result(x); + + // We use R4+R5 in order to get a temp effect. These regs are used in slow path (MonitorEnterStub). + LIR_Opr lock = FrameMap::R5_opr; + LIR_Opr scratch = FrameMap::R4_opr; + LIR_Opr hdr = FrameMap::R6_opr; + + CodeEmitInfo* info_for_exception = NULL; + if (x->needs_null_check()) { + info_for_exception = state_for(x); + } + + // This CodeEmitInfo must not have the xhandlers because here the + // object is already locked (xhandlers expects object to be unlocked). + CodeEmitInfo* info = state_for(x, x->state(), true); + monitor_enter(obj.result(), lock, hdr, scratch, x->monitor_no(), info_for_exception, info); +} + + +void LIRGenerator::do_MonitorExit(MonitorExit* x) { + assert(x->is_pinned(),""); + LIRItem obj(x->obj(), this); + obj.dont_load_item(); + + set_no_result(x); + LIR_Opr lock = FrameMap::R5_opr; + LIR_Opr hdr = FrameMap::R4_opr; // Used for slow path (MonitorExitStub). + LIR_Opr obj_temp = FrameMap::R6_opr; + monitor_exit(obj_temp, lock, hdr, LIR_OprFact::illegalOpr, x->monitor_no()); +} + + +// _ineg, _lneg, _fneg, _dneg +void LIRGenerator::do_NegateOp(NegateOp* x) { + LIRItem value(x->x(), this); + value.load_item(); + LIR_Opr reg = rlock_result(x); + __ negate(value.result(), reg); +} + + +// for _fadd, _fmul, _fsub, _fdiv, _frem +// _dadd, _dmul, _dsub, _ddiv, _drem +void LIRGenerator::do_ArithmeticOp_FPU(ArithmeticOp* x) { + switch (x->op()) { + case Bytecodes::_fadd: + case Bytecodes::_fmul: + case Bytecodes::_fsub: + case Bytecodes::_fdiv: + case Bytecodes::_dadd: + case Bytecodes::_dmul: + case Bytecodes::_dsub: + case Bytecodes::_ddiv: { + LIRItem left(x->x(), this); + LIRItem right(x->y(), this); + left.load_item(); + right.load_item(); + rlock_result(x); + arithmetic_op_fpu(x->op(), x->operand(), left.result(), right.result(), x->is_strictfp()); + } + break; + + case Bytecodes::_frem: + case Bytecodes::_drem: { + address entry = NULL; + switch (x->op()) { + case Bytecodes::_frem: + entry = CAST_FROM_FN_PTR(address, SharedRuntime::frem); + break; + case Bytecodes::_drem: + entry = CAST_FROM_FN_PTR(address, SharedRuntime::drem); + break; + default: + ShouldNotReachHere(); + } + LIR_Opr result = call_runtime(x->x(), x->y(), entry, x->type(), NULL); + set_result(x, result); + } + break; + + default: ShouldNotReachHere(); + } +} + + +// for _ladd, _lmul, _lsub, _ldiv, _lrem +void LIRGenerator::do_ArithmeticOp_Long(ArithmeticOp* x) { + bool is_div_rem = x->op() == Bytecodes::_ldiv || x->op() == Bytecodes::_lrem; + + LIRItem right(x->y(), this); + // Missing test if instr is commutative and if we should swap. + if (right.value()->type()->as_LongConstant() && + (x->op() == Bytecodes::_lsub && right.value()->type()->as_LongConstant()->value() == ((-1)<<15)) ) { + // Sub is implemented by addi and can't support min_simm16 as constant.. + right.load_item(); + } else { + right.load_nonconstant(); + } + assert(right.is_constant() || right.is_register(), "wrong state of right"); + + if (is_div_rem) { + LIR_Opr divisor = right.result(); + if (divisor->is_register()) { + CodeEmitInfo* null_check_info = state_for(x); + __ cmp(lir_cond_equal, divisor, LIR_OprFact::longConst(0)); + __ branch(lir_cond_equal, T_LONG, new DivByZeroStub(null_check_info)); + } else { + jlong const_divisor = divisor->as_constant_ptr()->as_jlong(); + if (const_divisor == 0) { + CodeEmitInfo* null_check_info = state_for(x); + __ jump(new DivByZeroStub(null_check_info)); + rlock_result(x); + __ move(LIR_OprFact::longConst(0), x->operand()); // dummy + return; + } + if (x->op() == Bytecodes::_lrem && !is_power_of_2(const_divisor) && const_divisor != -1) { + // Remainder computation would need additional tmp != R0. + right.load_item(); + } + } + } + + LIRItem left(x->x(), this); + left.load_item(); + rlock_result(x); + if (is_div_rem) { + CodeEmitInfo* info = NULL; // Null check already done above. + LIR_Opr tmp = FrameMap::R0_opr; + if (x->op() == Bytecodes::_lrem) { + __ irem(left.result(), right.result(), x->operand(), tmp, info); + } else if (x->op() == Bytecodes::_ldiv) { + __ idiv(left.result(), right.result(), x->operand(), tmp, info); + } + } else { + arithmetic_op_long(x->op(), x->operand(), left.result(), right.result(), NULL); + } +} + + +// for: _iadd, _imul, _isub, _idiv, _irem +void LIRGenerator::do_ArithmeticOp_Int(ArithmeticOp* x) { + bool is_div_rem = x->op() == Bytecodes::_idiv || x->op() == Bytecodes::_irem; + + LIRItem right(x->y(), this); + // Missing test if instr is commutative and if we should swap. + if (right.value()->type()->as_IntConstant() && + (x->op() == Bytecodes::_isub && right.value()->type()->as_IntConstant()->value() == ((-1)<<15)) ) { + // Sub is implemented by addi and can't support min_simm16 as constant. + right.load_item(); + } else { + right.load_nonconstant(); + } + assert(right.is_constant() || right.is_register(), "wrong state of right"); + + if (is_div_rem) { + LIR_Opr divisor = right.result(); + if (divisor->is_register()) { + CodeEmitInfo* null_check_info = state_for(x); + __ cmp(lir_cond_equal, divisor, LIR_OprFact::intConst(0)); + __ branch(lir_cond_equal, T_INT, new DivByZeroStub(null_check_info)); + } else { + jint const_divisor = divisor->as_constant_ptr()->as_jint(); + if (const_divisor == 0) { + CodeEmitInfo* null_check_info = state_for(x); + __ jump(new DivByZeroStub(null_check_info)); + rlock_result(x); + __ move(LIR_OprFact::intConst(0), x->operand()); // dummy + return; + } + if (x->op() == Bytecodes::_irem && !is_power_of_2(const_divisor) && const_divisor != -1) { + // Remainder computation would need additional tmp != R0. + right.load_item(); + } + } + } + + LIRItem left(x->x(), this); + left.load_item(); + rlock_result(x); + if (is_div_rem) { + CodeEmitInfo* info = NULL; // Null check already done above. + LIR_Opr tmp = FrameMap::R0_opr; + if (x->op() == Bytecodes::_irem) { + __ irem(left.result(), right.result(), x->operand(), tmp, info); + } else if (x->op() == Bytecodes::_idiv) { + __ idiv(left.result(), right.result(), x->operand(), tmp, info); + } + } else { + arithmetic_op_int(x->op(), x->operand(), left.result(), right.result(), FrameMap::R0_opr); + } +} + + +void LIRGenerator::do_ArithmeticOp(ArithmeticOp* x) { + ValueTag tag = x->type()->tag(); + assert(x->x()->type()->tag() == tag && x->y()->type()->tag() == tag, "wrong parameters"); + switch (tag) { + case floatTag: + case doubleTag: do_ArithmeticOp_FPU(x); return; + case longTag: do_ArithmeticOp_Long(x); return; + case intTag: do_ArithmeticOp_Int(x); return; + } + ShouldNotReachHere(); +} + + +// _ishl, _lshl, _ishr, _lshr, _iushr, _lushr +void LIRGenerator::do_ShiftOp(ShiftOp* x) { + LIRItem value(x->x(), this); + LIRItem count(x->y(), this); + value.load_item(); + LIR_Opr reg = rlock_result(x); + LIR_Opr mcount; + if (count.result()->is_register()) { + mcount = FrameMap::R0_opr; + } else { + mcount = LIR_OprFact::illegalOpr; + } + shift_op(x->op(), reg, value.result(), count.result(), mcount); +} + + +inline bool can_handle_logic_op_as_uimm(ValueType *type, Bytecodes::Code bc) { + jlong int_or_long_const; + if (type->as_IntConstant()) { + int_or_long_const = type->as_IntConstant()->value(); + } else if (type->as_LongConstant()) { + int_or_long_const = type->as_LongConstant()->value(); + } else if (type->as_ObjectConstant()) { + return type->as_ObjectConstant()->value()->is_null_object(); + } else { + return false; + } + + if (Assembler::is_uimm(int_or_long_const, 16)) return true; + if ((int_or_long_const & 0xFFFF) == 0 && + Assembler::is_uimm((jlong)((julong)int_or_long_const >> 16), 16)) return true; + + // see Assembler::andi + if (bc == Bytecodes::_iand && + (is_power_of_2_long(int_or_long_const+1) || + is_power_of_2_long(int_or_long_const) || + is_power_of_2_long(-int_or_long_const))) return true; + if (bc == Bytecodes::_land && + (is_power_of_2_long(int_or_long_const+1) || + (Assembler::is_uimm(int_or_long_const, 32) && is_power_of_2_long(int_or_long_const)) || + (int_or_long_const != min_jlong && is_power_of_2_long(-int_or_long_const)))) return true; + + // special case: xor -1 + if ((bc == Bytecodes::_ixor || bc == Bytecodes::_lxor) && + int_or_long_const == -1) return true; + return false; +} + + +// _iand, _land, _ior, _lor, _ixor, _lxor +void LIRGenerator::do_LogicOp(LogicOp* x) { + LIRItem left(x->x(), this); + LIRItem right(x->y(), this); + + left.load_item(); + + Value rval = right.value(); + LIR_Opr r = rval->operand(); + ValueType *type = rval->type(); + // Logic instructions use unsigned immediate values. + if (can_handle_logic_op_as_uimm(type, x->op())) { + if (!r->is_constant()) { + r = LIR_OprFact::value_type(type); + rval->set_operand(r); + } + right.set_result(r); + } else { + right.load_item(); + } + + LIR_Opr reg = rlock_result(x); + + logic_op(x->op(), reg, left.result(), right.result()); +} + + +// _lcmp, _fcmpl, _fcmpg, _dcmpl, _dcmpg +void LIRGenerator::do_CompareOp(CompareOp* x) { + LIRItem left(x->x(), this); + LIRItem right(x->y(), this); + left.load_item(); + right.load_item(); + LIR_Opr reg = rlock_result(x); + if (x->x()->type()->is_float_kind()) { + Bytecodes::Code code = x->op(); + __ fcmp2int(left.result(), right.result(), reg, (code == Bytecodes::_fcmpl || code == Bytecodes::_dcmpl)); + } else if (x->x()->type()->tag() == longTag) { + __ lcmp2int(left.result(), right.result(), reg); + } else { + Unimplemented(); + } +} + + +void LIRGenerator::do_CompareAndSwap(Intrinsic* x, ValueType* type) { + assert(x->number_of_arguments() == 4, "wrong type"); + LIRItem obj (x->argument_at(0), this); // object + LIRItem offset(x->argument_at(1), this); // offset of field + LIRItem cmp (x->argument_at(2), this); // Value to compare with field. + LIRItem val (x->argument_at(3), this); // Replace field with val if matches cmp. + + LIR_Opr t1 = LIR_OprFact::illegalOpr; + LIR_Opr t2 = LIR_OprFact::illegalOpr; + LIR_Opr addr = new_pointer_register(); + + // Get address of field. + obj.load_item(); + offset.load_item(); + cmp.load_item(); + val.load_item(); + + __ add(obj.result(), offset.result(), addr); + + // Volatile load may be followed by Unsafe CAS. + if (support_IRIW_for_not_multiple_copy_atomic_cpu) { + __ membar(); // To be safe. Unsafe semantics are unclear. + } else { + __ membar_release(); + } + + if (type == objectType) { // Write-barrier needed for Object fields. + // Only cmp value can get overwritten, no do_load required. + pre_barrier(LIR_OprFact::illegalOpr /* addr */, cmp.result() /* pre_val */, + false /* do_load */, false /* patch */, NULL); + } + + if (type == objectType) { + if (UseCompressedOops) { + t1 = new_register(T_OBJECT); + t2 = new_register(T_OBJECT); + } + __ cas_obj(addr, cmp.result(), val.result(), t1, t2); + } else if (type == intType) { + __ cas_int(addr, cmp.result(), val.result(), t1, t2); + } else if (type == longType) { + __ cas_long(addr, cmp.result(), val.result(), t1, t2); + } else { + ShouldNotReachHere(); + } + // Benerate conditional move of boolean result. + LIR_Opr result = rlock_result(x); + __ cmove(lir_cond_equal, LIR_OprFact::intConst(1), LIR_OprFact::intConst(0), + result, as_BasicType(type)); + if (type == objectType) { // Write-barrier needed for Object fields. + // Precise card mark since could either be object or array. + post_barrier(addr, val.result()); + } +} + + +void LIRGenerator::do_MathIntrinsic(Intrinsic* x) { + switch (x->id()) { + case vmIntrinsics::_dabs: { + assert(x->number_of_arguments() == 1, "wrong type"); + LIRItem value(x->argument_at(0), this); + value.load_item(); + LIR_Opr dst = rlock_result(x); + __ abs(value.result(), dst, LIR_OprFact::illegalOpr); + break; + } + case vmIntrinsics::_dsqrt: { + if (VM_Version::has_fsqrt()) { + assert(x->number_of_arguments() == 1, "wrong type"); + LIRItem value(x->argument_at(0), this); + value.load_item(); + LIR_Opr dst = rlock_result(x); + __ sqrt(value.result(), dst, LIR_OprFact::illegalOpr); + break; + } // else fallthru + } + case vmIntrinsics::_dlog10: // fall through + case vmIntrinsics::_dlog: // fall through + case vmIntrinsics::_dsin: // fall through + case vmIntrinsics::_dtan: // fall through + case vmIntrinsics::_dcos: // fall through + case vmIntrinsics::_dexp: { + assert(x->number_of_arguments() == 1, "wrong type"); + + address runtime_entry = NULL; + switch (x->id()) { + case vmIntrinsics::_dsqrt: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dsqrt); + break; + case vmIntrinsics::_dsin: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dsin); + break; + case vmIntrinsics::_dcos: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dcos); + break; + case vmIntrinsics::_dtan: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dtan); + break; + case vmIntrinsics::_dlog: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dlog); + break; + case vmIntrinsics::_dlog10: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dlog10); + break; + case vmIntrinsics::_dexp: + runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dexp); + break; + default: + ShouldNotReachHere(); + } + + LIR_Opr result = call_runtime(x->argument_at(0), runtime_entry, x->type(), NULL); + set_result(x, result); + break; + } + case vmIntrinsics::_dpow: { + assert(x->number_of_arguments() == 2, "wrong type"); + address runtime_entry = CAST_FROM_FN_PTR(address, SharedRuntime::dpow); + LIR_Opr result = call_runtime(x->argument_at(0), x->argument_at(1), runtime_entry, x->type(), NULL); + set_result(x, result); + break; + } + } +} + + +void LIRGenerator::do_ArrayCopy(Intrinsic* x) { + assert(x->number_of_arguments() == 5, "wrong type"); + + // Make all state_for calls early since they can emit code. + CodeEmitInfo* info = state_for(x, x->state()); + + LIRItem src (x->argument_at(0), this); + LIRItem src_pos (x->argument_at(1), this); + LIRItem dst (x->argument_at(2), this); + LIRItem dst_pos (x->argument_at(3), this); + LIRItem length (x->argument_at(4), this); + + // Load all values in callee_save_registers (C calling convention), + // as this makes the parameter passing to the fast case simpler. + src.load_item_force (FrameMap::R14_oop_opr); + src_pos.load_item_force (FrameMap::R15_opr); + dst.load_item_force (FrameMap::R17_oop_opr); + dst_pos.load_item_force (FrameMap::R18_opr); + length.load_item_force (FrameMap::R19_opr); + LIR_Opr tmp = FrameMap::R20_opr; + + int flags; + ciArrayKlass* expected_type; + arraycopy_helper(x, &flags, &expected_type); + + __ arraycopy(src.result(), src_pos.result(), dst.result(), dst_pos.result(), + length.result(), tmp, + expected_type, flags, info); + set_no_result(x); +} + + +// _i2l, _i2f, _i2d, _l2i, _l2f, _l2d, _f2i, _f2l, _f2d, _d2i, _d2l, _d2f +// _i2b, _i2c, _i2s +void LIRGenerator::do_Convert(Convert* x) { + switch (x->op()) { + + // int -> float: force spill + case Bytecodes::_l2f: { + if (!VM_Version::has_fcfids()) { // fcfids is >= Power7 only + // fcfid+frsp needs fixup code to avoid rounding incompatibility. + address entry = CAST_FROM_FN_PTR(address, SharedRuntime::l2f); + LIR_Opr result = call_runtime(x->value(), entry, x->type(), NULL); + set_result(x, result); + break; + } // else fallthru + } + case Bytecodes::_l2d: { + LIRItem value(x->value(), this); + LIR_Opr reg = rlock_result(x); + value.load_item(); + LIR_Opr tmp = force_to_spill(value.result(), T_DOUBLE); + __ convert(x->op(), tmp, reg); + break; + } + case Bytecodes::_i2f: + case Bytecodes::_i2d: { + LIRItem value(x->value(), this); + LIR_Opr reg = rlock_result(x); + value.load_item(); + // Convert i2l first. + LIR_Opr tmp1 = new_register(T_LONG); + __ convert(Bytecodes::_i2l, value.result(), tmp1); + LIR_Opr tmp2 = force_to_spill(tmp1, T_DOUBLE); + __ convert(x->op(), tmp2, reg); + break; + } + + // float -> int: result will be stored + case Bytecodes::_f2l: + case Bytecodes::_d2l: { + LIRItem value(x->value(), this); + LIR_Opr reg = rlock_result(x); + value.set_destroys_register(); // USE_KILL + value.load_item(); + set_vreg_flag(reg, must_start_in_memory); + __ convert(x->op(), value.result(), reg); + break; + } + case Bytecodes::_f2i: + case Bytecodes::_d2i: { + LIRItem value(x->value(), this); + LIR_Opr reg = rlock_result(x); + value.set_destroys_register(); // USE_KILL + value.load_item(); + // Convert l2i afterwards. + LIR_Opr tmp1 = new_register(T_LONG); + set_vreg_flag(tmp1, must_start_in_memory); + __ convert(x->op(), value.result(), tmp1); + __ convert(Bytecodes::_l2i, tmp1, reg); + break; + } + + // Within same category: just register conversions. + case Bytecodes::_i2b: + case Bytecodes::_i2c: + case Bytecodes::_i2s: + case Bytecodes::_i2l: + case Bytecodes::_l2i: + case Bytecodes::_f2d: + case Bytecodes::_d2f: { + LIRItem value(x->value(), this); + LIR_Opr reg = rlock_result(x); + value.load_item(); + __ convert(x->op(), value.result(), reg); + break; + } + + default: ShouldNotReachHere(); + } +} + + +void LIRGenerator::do_NewInstance(NewInstance* x) { + // This instruction can be deoptimized in the slow path. + const LIR_Opr reg = result_register_for(x->type()); +#ifndef PRODUCT + if (PrintNotLoaded && !x->klass()->is_loaded()) { + tty->print_cr(" ###class not loaded at new bci %d", x->printable_bci()); + } +#endif + CodeEmitInfo* info = state_for(x, x->state()); + LIR_Opr klass_reg = FrameMap::R4_metadata_opr; // Used by slow path (NewInstanceStub). + LIR_Opr tmp1 = FrameMap::R5_oop_opr; + LIR_Opr tmp2 = FrameMap::R6_oop_opr; + LIR_Opr tmp3 = FrameMap::R7_oop_opr; + LIR_Opr tmp4 = FrameMap::R8_oop_opr; + new_instance(reg, x->klass(), x->is_unresolved(), tmp1, tmp2, tmp3, tmp4, klass_reg, info); + + // Must prevent reordering of stores for object initialization + // with stores that publish the new object. + __ membar_storestore(); + LIR_Opr result = rlock_result(x); + __ move(reg, result); +} + + +void LIRGenerator::do_NewTypeArray(NewTypeArray* x) { + // Evaluate state_for early since it may emit code. + CodeEmitInfo* info = state_for(x, x->state()); + + LIRItem length(x->length(), this); + length.load_item(); + + LIR_Opr reg = result_register_for(x->type()); + LIR_Opr klass_reg = FrameMap::R4_metadata_opr; // Used by slow path (NewTypeArrayStub). + // We use R5 in order to get a temp effect. This reg is used in slow path (NewTypeArrayStub). + LIR_Opr tmp1 = FrameMap::R5_oop_opr; + LIR_Opr tmp2 = FrameMap::R6_oop_opr; + LIR_Opr tmp3 = FrameMap::R7_oop_opr; + LIR_Opr tmp4 = FrameMap::R8_oop_opr; + LIR_Opr len = length.result(); + BasicType elem_type = x->elt_type(); + + __ metadata2reg(ciTypeArrayKlass::make(elem_type)->constant_encoding(), klass_reg); + + CodeStub* slow_path = new NewTypeArrayStub(klass_reg, len, reg, info); + __ allocate_array(reg, len, tmp1, tmp2, tmp3, tmp4, elem_type, klass_reg, slow_path); + + // Must prevent reordering of stores for object initialization + // with stores that publish the new object. + __ membar_storestore(); + LIR_Opr result = rlock_result(x); + __ move(reg, result); +} + + +void LIRGenerator::do_NewObjectArray(NewObjectArray* x) { + // Evaluate state_for early since it may emit code. + CodeEmitInfo* info = state_for(x, x->state()); + // In case of patching (i.e., object class is not yet loaded), + // we need to reexecute the instruction and therefore provide + // the state before the parameters have been consumed. + CodeEmitInfo* patching_info = NULL; + if (!x->klass()->is_loaded() || PatchALot) { + patching_info = state_for(x, x->state_before()); + } + + LIRItem length(x->length(), this); + length.load_item(); + + const LIR_Opr reg = result_register_for(x->type()); + LIR_Opr klass_reg = FrameMap::R4_metadata_opr; // Used by slow path (NewObjectArrayStub). + // We use R5 in order to get a temp effect. This reg is used in slow path (NewObjectArrayStub). + LIR_Opr tmp1 = FrameMap::R5_oop_opr; + LIR_Opr tmp2 = FrameMap::R6_oop_opr; + LIR_Opr tmp3 = FrameMap::R7_oop_opr; + LIR_Opr tmp4 = FrameMap::R8_oop_opr; + LIR_Opr len = length.result(); + + CodeStub* slow_path = new NewObjectArrayStub(klass_reg, len, reg, info); + ciMetadata* obj = ciObjArrayKlass::make(x->klass()); + if (obj == ciEnv::unloaded_ciobjarrayklass()) { + BAILOUT("encountered unloaded_ciobjarrayklass due to out of memory error"); + } + klass2reg_with_patching(klass_reg, obj, patching_info); + __ allocate_array(reg, len, tmp1, tmp2, tmp3, tmp4, T_OBJECT, klass_reg, slow_path); + + // Must prevent reordering of stores for object initialization + // with stores that publish the new object. + __ membar_storestore(); + LIR_Opr result = rlock_result(x); + __ move(reg, result); +} + + +void LIRGenerator::do_NewMultiArray(NewMultiArray* x) { + Values* dims = x->dims(); + int i = dims->length(); + LIRItemList* items = new LIRItemList(dims->length(), NULL); + while (i-- > 0) { + LIRItem* size = new LIRItem(dims->at(i), this); + items->at_put(i, size); + } + + // Evaluate state_for early since it may emit code. + CodeEmitInfo* patching_info = NULL; + if (!x->klass()->is_loaded() || PatchALot) { + patching_info = state_for(x, x->state_before()); + + // Cannot re-use same xhandlers for multiple CodeEmitInfos, so + // clone all handlers (NOTE: Usually this is handled transparently + // by the CodeEmitInfo cloning logic in CodeStub constructors but + // is done explicitly here because a stub isn't being used). + x->set_exception_handlers(new XHandlers(x->exception_handlers())); + } + CodeEmitInfo* info = state_for(x, x->state()); + + i = dims->length(); + while (i-- > 0) { + LIRItem* size = items->at(i); + size->load_nonconstant(); + // FrameMap::_reserved_argument_area_size includes the dimensions + // varargs, because it's initialized to hir()->max_stack() when the + // FrameMap is created. + store_stack_parameter(size->result(), in_ByteSize(i*sizeof(jint) + FrameMap::first_available_sp_in_frame)); + } + + const LIR_Opr klass_reg = FrameMap::R4_metadata_opr; // Used by slow path. + klass2reg_with_patching(klass_reg, x->klass(), patching_info); + + LIR_Opr rank = FrameMap::R5_opr; // Used by slow path. + __ move(LIR_OprFact::intConst(x->rank()), rank); + + LIR_Opr varargs = FrameMap::as_pointer_opr(R6); // Used by slow path. + __ leal(LIR_OprFact::address(new LIR_Address(FrameMap::SP_opr, FrameMap::first_available_sp_in_frame, T_INT)), + varargs); + + // Note: This instruction can be deoptimized in the slow path. + LIR_OprList* args = new LIR_OprList(3); + args->append(klass_reg); + args->append(rank); + args->append(varargs); + const LIR_Opr reg = result_register_for(x->type()); + __ call_runtime(Runtime1::entry_for(Runtime1::new_multi_array_id), + LIR_OprFact::illegalOpr, + reg, args, info); + + // Must prevent reordering of stores for object initialization + // with stores that publish the new object. + __ membar_storestore(); + LIR_Opr result = rlock_result(x); + __ move(reg, result); +} + + +void LIRGenerator::do_BlockBegin(BlockBegin* x) { + // nothing to do for now +} + + +void LIRGenerator::do_CheckCast(CheckCast* x) { + LIRItem obj(x->obj(), this); + CodeEmitInfo* patching_info = NULL; + if (!x->klass()->is_loaded() || (PatchALot && !x->is_incompatible_class_change_check())) { + // Must do this before locking the destination register as + // an oop register, and before the obj is loaded (so x->obj()->item() + // is valid for creating a debug info location). + patching_info = state_for(x, x->state_before()); + } + obj.load_item(); + LIR_Opr out_reg = rlock_result(x); + CodeStub* stub; + CodeEmitInfo* info_for_exception = state_for(x); + + if (x->is_incompatible_class_change_check()) { + assert(patching_info == NULL, "can't patch this"); + stub = new SimpleExceptionStub(Runtime1::throw_incompatible_class_change_error_id, + LIR_OprFact::illegalOpr, info_for_exception); + } else { + stub = new SimpleExceptionStub(Runtime1::throw_class_cast_exception_id, obj.result(), info_for_exception); + } + // Following registers are used by slow_subtype_check: + LIR_Opr tmp1 = FrameMap::R4_oop_opr; // super_klass + LIR_Opr tmp2 = FrameMap::R5_oop_opr; // sub_klass + LIR_Opr tmp3 = FrameMap::R6_oop_opr; // temp + __ checkcast(out_reg, obj.result(), x->klass(), tmp1, tmp2, tmp3, + x->direct_compare(), info_for_exception, patching_info, stub, + x->profiled_method(), x->profiled_bci()); +} + + +void LIRGenerator::do_InstanceOf(InstanceOf* x) { + LIRItem obj(x->obj(), this); + CodeEmitInfo* patching_info = NULL; + if (!x->klass()->is_loaded() || PatchALot) { + patching_info = state_for(x, x->state_before()); + } + // Ensure the result register is not the input register because the + // result is initialized before the patching safepoint. + obj.load_item(); + LIR_Opr out_reg = rlock_result(x); + // Following registers are used by slow_subtype_check: + LIR_Opr tmp1 = FrameMap::R4_oop_opr; // super_klass + LIR_Opr tmp2 = FrameMap::R5_oop_opr; // sub_klass + LIR_Opr tmp3 = FrameMap::R6_oop_opr; // temp + __ instanceof(out_reg, obj.result(), x->klass(), tmp1, tmp2, tmp3, + x->direct_compare(), patching_info, + x->profiled_method(), x->profiled_bci()); +} + + +void LIRGenerator::do_If(If* x) { + assert(x->number_of_sux() == 2, "inconsistency"); + ValueTag tag = x->x()->type()->tag(); + LIRItem xitem(x->x(), this); + LIRItem yitem(x->y(), this); + LIRItem* xin = &xitem; + LIRItem* yin = &yitem; + If::Condition cond = x->cond(); + + LIR_Opr left = LIR_OprFact::illegalOpr; + LIR_Opr right = LIR_OprFact::illegalOpr; + + xin->load_item(); + left = xin->result(); + + if (yin->result()->is_constant() && yin->result()->type() == T_INT && + Assembler::is_simm16(yin->result()->as_constant_ptr()->as_jint())) { + // Inline int constants which are small enough to be immediate operands. + right = LIR_OprFact::value_type(yin->value()->type()); + } else if (tag == longTag && yin->is_constant() && yin->get_jlong_constant() == 0 && + (cond == If::eql || cond == If::neq)) { + // Inline long zero. + right = LIR_OprFact::value_type(yin->value()->type()); + } else if (tag == objectTag && yin->is_constant() && (yin->get_jobject_constant()->is_null_object())) { + right = LIR_OprFact::value_type(yin->value()->type()); + } else { + yin->load_item(); + right = yin->result(); + } + set_no_result(x); + + // Add safepoint before generating condition code so it can be recomputed. + if (x->is_safepoint()) { + // Increment backedge counter if needed. + increment_backedge_counter(state_for(x, x->state_before()), x->profiled_bci()); + __ safepoint(safepoint_poll_register(), state_for(x, x->state_before())); + } + + __ cmp(lir_cond(cond), left, right); + // Generate branch profiling. Profiling code doesn't kill flags. + profile_branch(x, cond); + move_to_phi(x->state()); + if (x->x()->type()->is_float_kind()) { + __ branch(lir_cond(cond), right->type(), x->tsux(), x->usux()); + } else { + __ branch(lir_cond(cond), right->type(), x->tsux()); + } + assert(x->default_sux() == x->fsux(), "wrong destination above"); + __ jump(x->default_sux()); +} + + +LIR_Opr LIRGenerator::getThreadPointer() { + return FrameMap::as_pointer_opr(R16_thread); +} + + +void LIRGenerator::trace_block_entry(BlockBegin* block) { + LIR_Opr arg1 = FrameMap::R3_opr; // ARG1 + __ move(LIR_OprFact::intConst(block->block_id()), arg1); + LIR_OprList* args = new LIR_OprList(1); + args->append(arg1); + address func = CAST_FROM_FN_PTR(address, Runtime1::trace_block_entry); + __ call_runtime_leaf(func, LIR_OprFact::illegalOpr, LIR_OprFact::illegalOpr, args); +} + + +void LIRGenerator::volatile_field_store(LIR_Opr value, LIR_Address* address, + CodeEmitInfo* info) { +#ifdef _LP64 + __ store(value, address, info); +#else + Unimplemented(); +// __ volatile_store_mem_reg(value, address, info); +#endif +} + +void LIRGenerator::volatile_field_load(LIR_Address* address, LIR_Opr result, + CodeEmitInfo* info) { +#ifdef _LP64 + __ load(address, result, info); +#else + Unimplemented(); +// __ volatile_load_mem_reg(address, result, info); +#endif +} + + +void LIRGenerator::put_Object_unsafe(LIR_Opr src, LIR_Opr offset, LIR_Opr data, + BasicType type, bool is_volatile) { + LIR_Opr base_op = src; + LIR_Opr index_op = offset; + + bool is_obj = (type == T_ARRAY || type == T_OBJECT); +#ifndef _LP64 + if (is_volatile && type == T_LONG) { + __ volatile_store_unsafe_reg(data, src, offset, type, NULL, lir_patch_none); + } else +#endif + { + if (type == T_BOOLEAN) { + type = T_BYTE; + } + LIR_Address* addr; + if (type == T_ARRAY || type == T_OBJECT) { + LIR_Opr tmp = new_pointer_register(); + __ add(base_op, index_op, tmp); + addr = new LIR_Address(tmp, type); + } else { + addr = new LIR_Address(base_op, index_op, type); + } + + if (is_obj) { + pre_barrier(LIR_OprFact::address(addr), LIR_OprFact::illegalOpr /* pre_val */, + true /* do_load */, false /* patch */, NULL); + // _bs->c1_write_barrier_pre(this, LIR_OprFact::address(addr)); + } + __ move(data, addr); + if (is_obj) { + // This address is precise. + post_barrier(LIR_OprFact::address(addr), data); + } + } +} + + +void LIRGenerator::get_Object_unsafe(LIR_Opr dst, LIR_Opr src, LIR_Opr offset, + BasicType type, bool is_volatile) { +#ifndef _LP64 + if (is_volatile && type == T_LONG) { + __ volatile_load_unsafe_reg(src, offset, dst, type, NULL, lir_patch_none); + } else +#endif + { + LIR_Address* addr = new LIR_Address(src, offset, type); + __ load(addr, dst); + } +} + + +void LIRGenerator::do_UnsafeGetAndSetObject(UnsafeGetAndSetObject* x) { + BasicType type = x->basic_type(); + LIRItem src(x->object(), this); + LIRItem off(x->offset(), this); + LIRItem value(x->value(), this); + + src.load_item(); + value.load_item(); + off.load_nonconstant(); + + LIR_Opr dst = rlock_result(x, type); + LIR_Opr data = value.result(); + bool is_obj = (type == T_ARRAY || type == T_OBJECT); + + LIR_Opr tmp = FrameMap::R0_opr; + LIR_Opr ptr = new_pointer_register(); + __ add(src.result(), off.result(), ptr); + + if (support_IRIW_for_not_multiple_copy_atomic_cpu) { + __ membar(); + } else { + __ membar_release(); + } + + if (x->is_add()) { + __ xadd(ptr, data, dst, tmp); + } else { + const bool can_move_barrier = true; // TODO: port GraphKit::can_move_pre_barrier() from C2 + if (!can_move_barrier && is_obj) { + // Do the pre-write barrier, if any. + pre_barrier(ptr, LIR_OprFact::illegalOpr /* pre_val */, + true /* do_load */, false /* patch */, NULL); + } + __ xchg(ptr, data, dst, tmp); + if (is_obj) { + // Seems to be a precise address. + post_barrier(ptr, data); + if (can_move_barrier) { + pre_barrier(LIR_OprFact::illegalOpr, dst /* pre_val */, + false /* do_load */, false /* patch */, NULL); + } + } + } + + __ membar(); +} + + +void LIRGenerator::do_update_CRC32(Intrinsic* x) { + assert(UseCRC32Intrinsics, "or should not be here"); + LIR_Opr result = rlock_result(x); + + switch (x->id()) { + case vmIntrinsics::_updateCRC32: { + LIRItem crc(x->argument_at(0), this); + LIRItem val(x->argument_at(1), this); + // Registers destroyed by update_crc32. + crc.set_destroys_register(); + val.set_destroys_register(); + crc.load_item(); + val.load_item(); + __ update_crc32(crc.result(), val.result(), result); + break; + } + case vmIntrinsics::_updateBytesCRC32: + case vmIntrinsics::_updateByteBufferCRC32: { + bool is_updateBytes = (x->id() == vmIntrinsics::_updateBytesCRC32); + + LIRItem crc(x->argument_at(0), this); + LIRItem buf(x->argument_at(1), this); + LIRItem off(x->argument_at(2), this); + LIRItem len(x->argument_at(3), this); + buf.load_item(); + off.load_nonconstant(); + + LIR_Opr index = off.result(); + int offset = is_updateBytes ? arrayOopDesc::base_offset_in_bytes(T_BYTE) : 0; + if (off.result()->is_constant()) { + index = LIR_OprFact::illegalOpr; + offset += off.result()->as_jint(); + } + LIR_Opr base_op = buf.result(); + LIR_Address* a = NULL; + + if (index->is_valid()) { + LIR_Opr tmp = new_register(T_LONG); + __ convert(Bytecodes::_i2l, index, tmp); + index = tmp; + __ add(index, LIR_OprFact::intptrConst(offset), index); + a = new LIR_Address(base_op, index, T_BYTE); + } else { + a = new LIR_Address(base_op, offset, T_BYTE); + } + + BasicTypeList signature(3); + signature.append(T_INT); + signature.append(T_ADDRESS); + signature.append(T_INT); + CallingConvention* cc = frame_map()->c_calling_convention(&signature); + const LIR_Opr result_reg = result_register_for(x->type()); + + LIR_Opr arg1 = cc->at(0), + arg2 = cc->at(1), + arg3 = cc->at(2); + + // CCallingConventionRequiresIntsAsLongs + crc.load_item_force(arg1); // We skip int->long conversion here, because CRC32 stub doesn't care about high bits. + __ leal(LIR_OprFact::address(a), arg2); + load_int_as_long(gen()->lir(), len, arg3); + + __ call_runtime_leaf(StubRoutines::updateBytesCRC32(), LIR_OprFact::illegalOpr, result_reg, cc->args()); + __ move(result_reg, result); + break; + } + default: { + ShouldNotReachHere(); + } + } +} diff --git a/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.cpp new file mode 100644 index 00000000000..7c73b1c70a8 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.cpp @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2005, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_Instruction.hpp" +#include "c1/c1_LinearScan.hpp" +#include "utilities/bitMap.inline.hpp" + +void LinearScan::allocate_fpu_stack() { + Unimplemented(); + // No FPU stack on PPC +} diff --git a/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.hpp new file mode 100644 index 00000000000..d0f6002929e --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_LinearScan_ppc.hpp @@ -0,0 +1,73 @@ +/* + * Copyright (c) 2005, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_LINEARSCAN_PPC_HPP +#define CPU_PPC_VM_C1_LINEARSCAN_PPC_HPP + +inline bool LinearScan::is_processed_reg_num(int reg_num) { + assert(FrameMap::R0_opr->cpu_regnr() == FrameMap::last_cpu_reg() + 1, "wrong assumption below"); + assert(FrameMap::R1_opr->cpu_regnr() == FrameMap::last_cpu_reg() + 2, "wrong assumption below"); + assert(FrameMap::R13_opr->cpu_regnr() == FrameMap::last_cpu_reg() + 3, "wrong assumption below"); + assert(FrameMap::R16_opr->cpu_regnr() == FrameMap::last_cpu_reg() + 4, "wrong assumption below"); + assert(FrameMap::R29_opr->cpu_regnr() == FrameMap::last_cpu_reg() + 5, "wrong assumption below"); + return reg_num <= FrameMap::last_cpu_reg() || reg_num >= pd_nof_cpu_regs_frame_map; +} + +inline int LinearScan::num_physical_regs(BasicType type) { + return 1; +} + + +inline bool LinearScan::requires_adjacent_regs(BasicType type) { + return false; +} + +inline bool LinearScan::is_caller_save(int assigned_reg) { + return true; // assigned_reg < pd_first_callee_saved_reg; +} + + +inline void LinearScan::pd_add_temps(LIR_Op* op) { + // No special case behaviours yet +} + + +inline bool LinearScanWalker::pd_init_regs_for_alloc(Interval* cur) { + if (allocator()->gen()->is_vreg_flag_set(cur->reg_num(), LIRGenerator::callee_saved)) { + assert(cur->type() != T_FLOAT && cur->type() != T_DOUBLE, "cpu regs only"); + _first_reg = pd_first_callee_saved_reg; + _last_reg = pd_last_callee_saved_reg; + ShouldNotReachHere(); // Currently no callee saved regs. + return true; + } else if (cur->type() == T_INT || cur->type() == T_LONG || cur->type() == T_OBJECT || + cur->type() == T_ADDRESS || cur->type() == T_METADATA) { + _first_reg = pd_first_cpu_reg; + _last_reg = pd_last_cpu_reg; + return true; + } + return false; +} + +#endif // CPU_PPC_VM_C1_LINEARSCAN_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.cpp new file mode 100644 index 00000000000..f41a83b1c71 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.cpp @@ -0,0 +1,486 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_MacroAssembler.hpp" +#include "c1/c1_Runtime1.hpp" +#include "classfile/systemDictionary.hpp" +#include "gc/shared/collectedHeap.hpp" +#include "interpreter/interpreter.hpp" +#include "oops/arrayOop.hpp" +#include "oops/markOop.hpp" +#include "runtime/basicLock.hpp" +#include "runtime/biasedLocking.hpp" +#include "runtime/os.hpp" +#include "runtime/stubRoutines.hpp" +#include "runtime/sharedRuntime.hpp" + + +void C1_MacroAssembler::inline_cache_check(Register receiver, Register iCache) { + const Register temp_reg = R12_scratch2; + verify_oop(receiver); + load_klass(temp_reg, receiver); + if (TrapBasedICMissChecks) { + trap_ic_miss_check(temp_reg, iCache); + } else { + Label L; + cmpd(CCR0, temp_reg, iCache); + beq(CCR0, L); + //load_const_optimized(temp_reg, SharedRuntime::get_ic_miss_stub(), R0); + calculate_address_from_global_toc(temp_reg, SharedRuntime::get_ic_miss_stub(), true, true, false); + mtctr(temp_reg); + bctr(); + align(32, 12); + bind(L); + } +} + + +void C1_MacroAssembler::explicit_null_check(Register base) { + Unimplemented(); +} + + +void C1_MacroAssembler::build_frame(int frame_size_in_bytes, int bang_size_in_bytes) { + assert(bang_size_in_bytes >= frame_size_in_bytes, "stack bang size incorrect"); + // Make sure there is enough stack space for this method's activation. + generate_stack_overflow_check(bang_size_in_bytes); + + // Create the frame. + const Register return_pc = R0; + + mflr(return_pc); + // Get callers sp. + std(return_pc, _abi(lr), R1_SP); // SP->lr = return_pc + push_frame(frame_size_in_bytes, R0); // SP -= frame_size_in_bytes +} + + +void C1_MacroAssembler::unverified_entry(Register receiver, Register ic_klass) { + Unimplemented(); // Currently unused. + //if (C1Breakpoint) illtrap(); + //inline_cache_check(receiver, ic_klass); +} + + +void C1_MacroAssembler::verified_entry() { + if (C1Breakpoint) illtrap(); + // build frame +} + + +void C1_MacroAssembler::lock_object(Register Rmark, Register Roop, Register Rbox, Register Rscratch, Label& slow_case) { + assert_different_registers(Rmark, Roop, Rbox, Rscratch); + + Label done, cas_failed, slow_int; + + // The following move must be the first instruction of emitted since debug + // information may be generated for it. + // Load object header. + ld(Rmark, oopDesc::mark_offset_in_bytes(), Roop); + + verify_oop(Roop); + + // Save object being locked into the BasicObjectLock... + std(Roop, BasicObjectLock::obj_offset_in_bytes(), Rbox); + + if (UseBiasedLocking) { + biased_locking_enter(CCR0, Roop, Rmark, Rscratch, R0, done, &slow_int); + } + + // ... and mark it unlocked. + ori(Rmark, Rmark, markOopDesc::unlocked_value); + + // Save unlocked object header into the displaced header location on the stack. + std(Rmark, BasicLock::displaced_header_offset_in_bytes(), Rbox); + + // Compare object markOop with Rmark and if equal exchange Rscratch with object markOop. + assert(oopDesc::mark_offset_in_bytes() == 0, "cas must take a zero displacement"); + cmpxchgd(/*flag=*/CCR0, + /*current_value=*/Rscratch, + /*compare_value=*/Rmark, + /*exchange_value=*/Rbox, + /*where=*/Roop/*+0==mark_offset_in_bytes*/, + MacroAssembler::MemBarRel | MacroAssembler::MemBarAcq, + MacroAssembler::cmpxchgx_hint_acquire_lock(), + noreg, + &cas_failed, + /*check without membar and ldarx first*/true); + // If compare/exchange succeeded we found an unlocked object and we now have locked it + // hence we are done. + b(done); + + bind(slow_int); + b(slow_case); // far + + bind(cas_failed); + // We did not find an unlocked object so see if this is a recursive case. + sub(Rscratch, Rscratch, R1_SP); + load_const_optimized(R0, (~(os::vm_page_size()-1) | markOopDesc::lock_mask_in_place)); + and_(R0/*==0?*/, Rscratch, R0); + std(R0/*==0, perhaps*/, BasicLock::displaced_header_offset_in_bytes(), Rbox); + bne(CCR0, slow_int); + + bind(done); +} + + +void C1_MacroAssembler::unlock_object(Register Rmark, Register Roop, Register Rbox, Label& slow_case) { + assert_different_registers(Rmark, Roop, Rbox); + + Label slow_int, done; + + Address mark_addr(Roop, oopDesc::mark_offset_in_bytes()); + assert(mark_addr.disp() == 0, "cas must take a zero displacement"); + + if (UseBiasedLocking) { + // Load the object out of the BasicObjectLock. + ld(Roop, BasicObjectLock::obj_offset_in_bytes(), Rbox); + verify_oop(Roop); + biased_locking_exit(CCR0, Roop, R0, done); + } + // Test first it it is a fast recursive unlock. + ld(Rmark, BasicLock::displaced_header_offset_in_bytes(), Rbox); + cmpdi(CCR0, Rmark, 0); + beq(CCR0, done); + if (!UseBiasedLocking) { + // Load object. + ld(Roop, BasicObjectLock::obj_offset_in_bytes(), Rbox); + verify_oop(Roop); + } + + // Check if it is still a light weight lock, this is is true if we see + // the stack address of the basicLock in the markOop of the object. + cmpxchgd(/*flag=*/CCR0, + /*current_value=*/R0, + /*compare_value=*/Rbox, + /*exchange_value=*/Rmark, + /*where=*/Roop, + MacroAssembler::MemBarRel, + MacroAssembler::cmpxchgx_hint_release_lock(), + noreg, + &slow_int); + b(done); + bind(slow_int); + b(slow_case); // far + + // Done + bind(done); +} + + +void C1_MacroAssembler::try_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register, must be global register for incr_allocated_bytes + Register t2, // temp register + Label& slow_case // continuation point if fast allocation fails +) { + if (UseTLAB) { + tlab_allocate(obj, var_size_in_bytes, con_size_in_bytes, t1, slow_case); + } else { + eden_allocate(obj, var_size_in_bytes, con_size_in_bytes, t1, t2, slow_case); + RegisterOrConstant size_in_bytes = var_size_in_bytes->is_valid() + ? RegisterOrConstant(var_size_in_bytes) + : RegisterOrConstant(con_size_in_bytes); + incr_allocated_bytes(size_in_bytes, t1, t2); + } +} + + +void C1_MacroAssembler::initialize_header(Register obj, Register klass, Register len, Register t1, Register t2) { + assert_different_registers(obj, klass, len, t1, t2); + if (UseBiasedLocking && !len->is_valid()) { + ld(t1, in_bytes(Klass::prototype_header_offset()), klass); + } else { + load_const_optimized(t1, (intx)markOopDesc::prototype()); + } + std(t1, oopDesc::mark_offset_in_bytes(), obj); + store_klass(obj, klass); + if (len->is_valid()) { + stw(len, arrayOopDesc::length_offset_in_bytes(), obj); + } else if (UseCompressedClassPointers) { + // Otherwise length is in the class gap. + store_klass_gap(obj); + } +} + + +void C1_MacroAssembler::initialize_body(Register base, Register index) { + assert_different_registers(base, index); + srdi(index, index, LogBytesPerWord); + clear_memory_doubleword(base, index); +} + +void C1_MacroAssembler::initialize_body(Register obj, Register tmp1, Register tmp2, + int obj_size_in_bytes, int hdr_size_in_bytes) { + const int index = (obj_size_in_bytes - hdr_size_in_bytes) / HeapWordSize; + + const int cl_size = VM_Version::L1_data_cache_line_size(), + cl_dwords = cl_size>>3, + cl_dw_addr_bits = exact_log2(cl_dwords); + + const Register tmp = R0, + base_ptr = tmp1, + cnt_dwords = tmp2; + + if (index <= 6) { + // Use explicit NULL stores. + if (index > 0) { li(tmp, 0); } + for (int i = 0; i < index; ++i) { std(tmp, hdr_size_in_bytes + i * HeapWordSize, obj); } + + } else if (index < (2<0). + andi(cnt_dwords, cnt_dwords, cl_dwords-1); // Rest in dwords. + mtctr(tmp); // Load counter. + + bind(fastloop); + dcbz(base_ptr); // Clear 128byte aligned block. + addi(base_ptr, base_ptr, cl_size); + bdnz(fastloop); + + cmpdi(CCR0, cnt_dwords, 0); // size 0? + beq(CCR0, done); // rest == 0 + li(tmp, 0); + mtctr(cnt_dwords); // Load counter. + + bind(restloop); // Clear rest. + std(tmp, 0, base_ptr); // Clear 8byte aligned block. + addi(base_ptr, base_ptr, 8); + bdnz(restloop); + + bind(done); + } +} + +void C1_MacroAssembler::allocate_object( + Register obj, // result: pointer to object after successful allocation + Register t1, // temp register + Register t2, // temp register + Register t3, // temp register + int hdr_size, // object header size in words + int obj_size, // object size in words + Register klass, // object klass + Label& slow_case // continuation point if fast allocation fails +) { + assert_different_registers(obj, t1, t2, t3, klass); + + // allocate space & initialize header + if (!is_simm16(obj_size * wordSize)) { + // Would need to use extra register to load + // object size => go the slow case for now. + b(slow_case); + return; + } + try_allocate(obj, noreg, obj_size * wordSize, t2, t3, slow_case); + + initialize_object(obj, klass, noreg, obj_size * HeapWordSize, t1, t2); +} + +void C1_MacroAssembler::initialize_object( + Register obj, // result: pointer to object after successful allocation + Register klass, // object klass + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Register t2 // temp register + ) { + const int hdr_size_in_bytes = instanceOopDesc::header_size() * HeapWordSize; + + initialize_header(obj, klass, noreg, t1, t2); + +#ifdef ASSERT + { + lwz(t1, in_bytes(Klass::layout_helper_offset()), klass); + if (var_size_in_bytes != noreg) { + cmpw(CCR0, t1, var_size_in_bytes); + } else { + cmpwi(CCR0, t1, con_size_in_bytes); + } + asm_assert_eq("bad size in initialize_object", 0x753); + } +#endif + + // Initialize body. + if (var_size_in_bytes != noreg) { + // Use a loop. + addi(t1, obj, hdr_size_in_bytes); // Compute address of first element. + addi(t2, var_size_in_bytes, -hdr_size_in_bytes); // Compute size of body. + initialize_body(t1, t2); + } else if (con_size_in_bytes > hdr_size_in_bytes) { + // Use a loop. + initialize_body(obj, t1, t2, con_size_in_bytes, hdr_size_in_bytes); + } + + if (CURRENT_ENV->dtrace_alloc_probes()) { + Unimplemented(); +// assert(obj == O0, "must be"); +// call(CAST_FROM_FN_PTR(address, Runtime1::entry_for(Runtime1::dtrace_object_alloc_id)), +// relocInfo::runtime_call_type); + } + + verify_oop(obj); +} + + +void C1_MacroAssembler::allocate_array( + Register obj, // result: pointer to array after successful allocation + Register len, // array length + Register t1, // temp register + Register t2, // temp register + Register t3, // temp register + int hdr_size, // object header size in words + int elt_size, // element size in bytes + Register klass, // object klass + Label& slow_case // continuation point if fast allocation fails +) { + assert_different_registers(obj, len, t1, t2, t3, klass); + + // Determine alignment mask. + assert(!(BytesPerWord & 1), "must be a multiple of 2 for masking code to work"); + int log2_elt_size = exact_log2(elt_size); + + // Check for negative or excessive length. + size_t max_length = max_array_allocation_length >> log2_elt_size; + if (UseTLAB) { + size_t max_tlab = align_size_up(ThreadLocalAllocBuffer::max_size() >> log2_elt_size, 64*K); + if (max_tlab < max_length) { max_length = max_tlab; } + } + load_const_optimized(t1, max_length); + cmpld(CCR0, len, t1); + bc_far_optimized(Assembler::bcondCRbiIs1, bi0(CCR0, Assembler::greater), slow_case); + + // compute array size + // note: If 0 <= len <= max_length, len*elt_size + header + alignment is + // smaller or equal to the largest integer; also, since top is always + // aligned, we can do the alignment here instead of at the end address + // computation. + const Register arr_size = t1; + Register arr_len_in_bytes = len; + if (elt_size != 1) { + sldi(t1, len, log2_elt_size); + arr_len_in_bytes = t1; + } + addi(arr_size, arr_len_in_bytes, hdr_size * wordSize + MinObjAlignmentInBytesMask); // Add space for header & alignment. + clrrdi(arr_size, arr_size, LogMinObjAlignmentInBytes); // Align array size. + + // Allocate space & initialize header. + if (UseTLAB) { + tlab_allocate(obj, arr_size, 0, t2, slow_case); + } else { + eden_allocate(obj, arr_size, 0, t2, t3, slow_case); + } + initialize_header(obj, klass, len, t2, t3); + + // Initialize body. + const Register base = t2; + const Register index = t3; + addi(base, obj, hdr_size * wordSize); // compute address of first element + addi(index, arr_size, -(hdr_size * wordSize)); // compute index = number of bytes to clear + initialize_body(base, index); + + if (CURRENT_ENV->dtrace_alloc_probes()) { + Unimplemented(); + //assert(obj == O0, "must be"); + //call(CAST_FROM_FN_PTR(address, Runtime1::entry_for(Runtime1::dtrace_object_alloc_id)), + // relocInfo::runtime_call_type); + } + + verify_oop(obj); +} + + +#ifndef PRODUCT + +void C1_MacroAssembler::verify_stack_oop(int stack_offset) { + verify_oop_addr((RegisterOrConstant)(stack_offset + STACK_BIAS), R1_SP, "broken oop in stack slot"); +} + +void C1_MacroAssembler::verify_not_null_oop(Register r) { + Label not_null; + cmpdi(CCR0, r, 0); + bne(CCR0, not_null); + stop("non-null oop required"); + bind(not_null); + if (!VerifyOops) return; + verify_oop(r); +} + +#endif // PRODUCT + +void C1_MacroAssembler::null_check(Register r, Label* Lnull) { + if (TrapBasedNullChecks) { // SIGTRAP based + trap_null_check(r); + } else { // explicit + //const address exception_entry = Runtime1::entry_for(Runtime1::throw_null_pointer_exception_id); + assert(Lnull != NULL, "must have Label for explicit check"); + cmpdi(CCR0, r, 0); + bc_far_optimized(Assembler::bcondCRbiIs1, bi0(CCR0, Assembler::equal), *Lnull); + } +} + +address C1_MacroAssembler::call_c_with_frame_resize(address dest, int frame_resize) { + if (frame_resize) { resize_frame(-frame_resize, R0); } +#if defined(ABI_ELFv2) + address return_pc = call_c(dest, relocInfo::runtime_call_type); +#else + address return_pc = call_c(CAST_FROM_FN_PTR(FunctionDescriptor*, dest), relocInfo::runtime_call_type); +#endif + if (frame_resize) { resize_frame(frame_resize, R0); } + return return_pc; +} diff --git a/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.hpp new file mode 100644 index 00000000000..9989baa8700 --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_MacroAssembler_ppc.hpp @@ -0,0 +1,93 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_MACROASSEMBLER_PPC_HPP +#define CPU_PPC_VM_C1_MACROASSEMBLER_PPC_HPP + + void pd_init() { /* nothing to do */ } + + public: + void try_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Register t2, // temp register + Label& slow_case // continuation point if fast allocation fails + ); + + void initialize_header(Register obj, Register klass, Register len, Register t1, Register t2); + void initialize_body(Register base, Register index); + void initialize_body(Register obj, Register tmp1, Register tmp2, int obj_size_in_bytes, int hdr_size_in_bytes); + + // locking/unlocking + void lock_object (Register Rmark, Register Roop, Register Rbox, Register Rscratch, Label& slow_case); + void unlock_object(Register Rmark, Register Roop, Register Rbox, Label& slow_case); + + void initialize_object( + Register obj, // result: pointer to object after successful allocation + Register klass, // object klass + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Register t2 // temp register + ); + + // Allocation of fixed-size objects + // (Can also be used to allocate fixed-size arrays, by setting + // hdr_size correctly and storing the array length afterwards.) + void allocate_object( + Register obj, // result: pointer to object after successful allocation + Register t1, // temp register + Register t2, // temp register + Register t3, // temp register + int hdr_size, // object header size in words + int obj_size, // object size in words + Register klass, // object klass + Label& slow_case // continuation point if fast allocation fails + ); + + enum { + max_array_allocation_length = 0x40000000 // ppc friendly value, requires lis only + }; + + // Allocation of arrays + void allocate_array( + Register obj, // result: pointer to array after successful allocation + Register len, // array length + Register t1, // temp register + Register t2, // temp register + Register t3, // temp register + int hdr_size, // object header size in words + int elt_size, // element size in bytes + Register klass, // object klass + Label& slow_case // continuation point if fast allocation fails + ); + + void null_check(Register r, Label *Lnull = NULL); + + address call_c_with_frame_resize(address dest, int frame_resize); + +#endif // CPU_PPC_VM_C1_MACROASSEMBLER_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c1_Runtime1_ppc.cpp b/hotspot/src/cpu/ppc/vm/c1_Runtime1_ppc.cpp new file mode 100644 index 00000000000..5fbaa4beb4e --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_Runtime1_ppc.cpp @@ -0,0 +1,1020 @@ +/* + * Copyright (c) 1999, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#include "precompiled.hpp" +#include "c1/c1_Defs.hpp" +#include "c1/c1_MacroAssembler.hpp" +#include "c1/c1_Runtime1.hpp" +#include "interpreter/interpreter.hpp" +#include "nativeInst_ppc.hpp" +#include "oops/compiledICHolder.hpp" +#include "oops/oop.inline.hpp" +#include "prims/jvmtiExport.hpp" +#include "register_ppc.hpp" +#include "runtime/sharedRuntime.hpp" +#include "runtime/signature.hpp" +#include "runtime/vframeArray.hpp" +#include "utilities/macros.hpp" +#include "vmreg_ppc.inline.hpp" +#if INCLUDE_ALL_GCS +#include "gc/g1/g1SATBCardTableModRefBS.hpp" +#endif + +// Implementation of StubAssembler + +int StubAssembler::call_RT(Register oop_result1, Register metadata_result, + address entry_point, int number_of_arguments) { + set_num_rt_args(0); // Nothing on stack + assert(!(oop_result1->is_valid() || metadata_result->is_valid()) || + oop_result1 != metadata_result, "registers must be different"); + + // Currently no stack banging. We assume that there are enough + // StackShadowPages (which have been banged in generate_stack_overflow_check) + // for the stub frame and the runtime frames. + + set_last_Java_frame(R1_SP, noreg); + + // ARG1 must hold thread address. + mr(R3_ARG1, R16_thread); + + address return_pc = call_c_with_frame_resize(entry_point, /*No resize, we have a C compatible frame.*/0); + + reset_last_Java_frame(); + + // Check for pending exceptions. + { + ld(R0, in_bytes(Thread::pending_exception_offset()), R16_thread); + cmpdi(CCR0, R0, 0); + + // This used to conditionally jump to forward_exception however it is + // possible if we relocate that the branch will not reach. So we must jump + // around so we can always reach. + + Label ok; + beq(CCR0, ok); + + // Make sure that the vm_results are cleared. + if (oop_result1->is_valid() || metadata_result->is_valid()) { + li(R0, 0); + if (oop_result1->is_valid()) { + std(R0, in_bytes(JavaThread::vm_result_offset()), R16_thread); + } + if (metadata_result->is_valid()) { + std(R0, in_bytes(JavaThread::vm_result_2_offset()), R16_thread); + } + } + + if (frame_size() == no_frame_size) { + ShouldNotReachHere(); // We always have a frame size. + //pop_frame(); // pop the stub frame + //ld(R0, _abi(lr), R1_SP); + //mtlr(R0); + //load_const_optimized(R0, StubRoutines::forward_exception_entry()); + //mtctr(R0); + //bctr(); + } else if (_stub_id == Runtime1::forward_exception_id) { + should_not_reach_here(); + } else { + // keep stub frame for next call_RT + //load_const_optimized(R0, Runtime1::entry_for(Runtime1::forward_exception_id)); + add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(Runtime1::entry_for(Runtime1::forward_exception_id))); + mtctr(R0); + bctr(); + } + + bind(ok); + } + + // Get oop results if there are any and reset the values in the thread. + if (oop_result1->is_valid()) { + get_vm_result(oop_result1); + } + if (metadata_result->is_valid()) { + get_vm_result_2(metadata_result); + } + + return (int)(return_pc - code_section()->start()); +} + + +int StubAssembler::call_RT(Register oop_result1, Register metadata_result, address entry, Register arg1) { + mr_if_needed(R4_ARG2, arg1); + return call_RT(oop_result1, metadata_result, entry, 1); +} + + +int StubAssembler::call_RT(Register oop_result1, Register metadata_result, address entry, Register arg1, Register arg2) { + mr_if_needed(R4_ARG2, arg1); + mr_if_needed(R5_ARG3, arg2); assert(arg2 != R4_ARG2, "smashed argument"); + return call_RT(oop_result1, metadata_result, entry, 2); +} + + +int StubAssembler::call_RT(Register oop_result1, Register metadata_result, address entry, Register arg1, Register arg2, Register arg3) { + mr_if_needed(R4_ARG2, arg1); + mr_if_needed(R5_ARG3, arg2); assert(arg2 != R4_ARG2, "smashed argument"); + mr_if_needed(R6_ARG4, arg3); assert(arg3 != R4_ARG2 && arg3 != R5_ARG3, "smashed argument"); + return call_RT(oop_result1, metadata_result, entry, 3); +} + + +// Implementation of Runtime1 + +#define __ sasm-> + +static int cpu_reg_save_offsets[FrameMap::nof_cpu_regs]; +static int fpu_reg_save_offsets[FrameMap::nof_fpu_regs]; +static int frame_size_in_bytes = -1; + +static OopMap* generate_oop_map(StubAssembler* sasm, bool save_fpu_registers) { + assert(frame_size_in_bytes > frame::abi_reg_args_size, "init"); + sasm->set_frame_size(frame_size_in_bytes / BytesPerWord); + int frame_size_in_slots = frame_size_in_bytes / sizeof(jint); + OopMap* oop_map = new OopMap(frame_size_in_slots, 0); + + int i; + for (i = 0; i < FrameMap::nof_cpu_regs; i++) { + Register r = as_Register(i); + if (FrameMap::reg_needs_save(r)) { + int sp_offset = cpu_reg_save_offsets[i]; + oop_map->set_callee_saved(VMRegImpl::stack2reg(sp_offset>>2), r->as_VMReg()); + oop_map->set_callee_saved(VMRegImpl::stack2reg((sp_offset>>2) + 1), r->as_VMReg()->next()); + } + } + + if (save_fpu_registers) { + for (i = 0; i < FrameMap::nof_fpu_regs; i++) { + FloatRegister r = as_FloatRegister(i); + int sp_offset = fpu_reg_save_offsets[i]; + oop_map->set_callee_saved(VMRegImpl::stack2reg(sp_offset>>2), r->as_VMReg()); + oop_map->set_callee_saved(VMRegImpl::stack2reg((sp_offset>>2) + 1), r->as_VMReg()->next()); + } + } + + return oop_map; +} + +static OopMap* save_live_registers(StubAssembler* sasm, bool save_fpu_registers = true, + Register ret_pc = noreg, int stack_preserve = 0) { + if (ret_pc == noreg) { + ret_pc = R0; + __ mflr(ret_pc); + } + __ std(ret_pc, _abi(lr), R1_SP); // C code needs pc in C1 method. + __ push_frame(frame_size_in_bytes + stack_preserve, R0); + + // Record volatile registers as callee-save values in an OopMap so + // their save locations will be propagated to the caller frame's + // RegisterMap during StackFrameStream construction (needed for + // deoptimization; see compiledVFrame::create_stack_value). + // OopMap frame sizes are in c2 stack slot sizes (sizeof(jint)). + + int i; + for (i = 0; i < FrameMap::nof_cpu_regs; i++) { + Register r = as_Register(i); + if (FrameMap::reg_needs_save(r)) { + int sp_offset = cpu_reg_save_offsets[i]; + __ std(r, sp_offset + STACK_BIAS, R1_SP); + } + } + + if (save_fpu_registers) { + for (i = 0; i < FrameMap::nof_fpu_regs; i++) { + FloatRegister r = as_FloatRegister(i); + int sp_offset = fpu_reg_save_offsets[i]; + __ stfd(r, sp_offset + STACK_BIAS, R1_SP); + } + } + + return generate_oop_map(sasm, save_fpu_registers); +} + +static void restore_live_registers(StubAssembler* sasm, Register result1, Register result2, + bool restore_fpu_registers = true) { + for (int i = 0; i < FrameMap::nof_cpu_regs; i++) { + Register r = as_Register(i); + if (FrameMap::reg_needs_save(r) && r != result1 && r != result2) { + int sp_offset = cpu_reg_save_offsets[i]; + __ ld(r, sp_offset + STACK_BIAS, R1_SP); + } + } + + if (restore_fpu_registers) { + for (int i = 0; i < FrameMap::nof_fpu_regs; i++) { + FloatRegister r = as_FloatRegister(i); + int sp_offset = fpu_reg_save_offsets[i]; + __ lfd(r, sp_offset + STACK_BIAS, R1_SP); + } + } + + __ pop_frame(); + __ ld(R0, _abi(lr), R1_SP); + __ mtlr(R0); +} + + +void Runtime1::initialize_pd() { + int i; + int sp_offset = frame::abi_reg_args_size; + + for (i = 0; i < FrameMap::nof_cpu_regs; i++) { + Register r = as_Register(i); + if (FrameMap::reg_needs_save(r)) { + cpu_reg_save_offsets[i] = sp_offset; + sp_offset += BytesPerWord; + } + } + + for (i = 0; i < FrameMap::nof_fpu_regs; i++) { + fpu_reg_save_offsets[i] = sp_offset; + sp_offset += BytesPerWord; + } + frame_size_in_bytes = align_size_up(sp_offset, frame::alignment_in_bytes); +} + + +OopMapSet* Runtime1::generate_exception_throw(StubAssembler* sasm, address target, bool has_argument) { + // Make a frame and preserve the caller's caller-save registers. + OopMap* oop_map = save_live_registers(sasm); + + int call_offset; + if (!has_argument) { + call_offset = __ call_RT(noreg, noreg, target); + } else { + call_offset = __ call_RT(noreg, noreg, target, R4_ARG2); + } + OopMapSet* oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + __ should_not_reach_here(); + return oop_maps; +} + +static OopMapSet* generate_exception_throw_with_stack_parms(StubAssembler* sasm, address target, + int stack_parms) { + // Make a frame and preserve the caller's caller-save registers. + const int parm_size_in_bytes = align_size_up(stack_parms << LogBytesPerWord, frame::alignment_in_bytes); + const int padding = parm_size_in_bytes - (stack_parms << LogBytesPerWord); + OopMap* oop_map = save_live_registers(sasm, true, noreg, parm_size_in_bytes); + + int call_offset = 0; + switch (stack_parms) { + case 3: + __ ld(R6_ARG4, frame_size_in_bytes + padding + 16, R1_SP); + case 2: + __ ld(R5_ARG3, frame_size_in_bytes + padding + 8, R1_SP); + case 1: + __ ld(R4_ARG2, frame_size_in_bytes + padding + 0, R1_SP); + call_offset = __ call_RT(noreg, noreg, target); + break; + default: Unimplemented(); break; + } + OopMapSet* oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + __ should_not_reach_here(); + return oop_maps; +} + + +OopMapSet* Runtime1::generate_stub_call(StubAssembler* sasm, Register result, address target, + Register arg1, Register arg2, Register arg3) { + // Make a frame and preserve the caller's caller-save registers. + OopMap* oop_map = save_live_registers(sasm); + + int call_offset; + if (arg1 == noreg) { + call_offset = __ call_RT(result, noreg, target); + } else if (arg2 == noreg) { + call_offset = __ call_RT(result, noreg, target, arg1); + } else if (arg3 == noreg) { + call_offset = __ call_RT(result, noreg, target, arg1, arg2); + } else { + call_offset = __ call_RT(result, noreg, target, arg1, arg2, arg3); + } + OopMapSet* oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + restore_live_registers(sasm, result, noreg); + __ blr(); + return oop_maps; +} + +static OopMapSet* stub_call_with_stack_parms(StubAssembler* sasm, Register result, address target, + int stack_parms, bool do_return = true) { + // Make a frame and preserve the caller's caller-save registers. + const int parm_size_in_bytes = align_size_up(stack_parms << LogBytesPerWord, frame::alignment_in_bytes); + const int padding = parm_size_in_bytes - (stack_parms << LogBytesPerWord); + OopMap* oop_map = save_live_registers(sasm, true, noreg, parm_size_in_bytes); + + int call_offset = 0; + switch (stack_parms) { + case 3: + __ ld(R6_ARG4, frame_size_in_bytes + padding + 16, R1_SP); + case 2: + __ ld(R5_ARG3, frame_size_in_bytes + padding + 8, R1_SP); + case 1: + __ ld(R4_ARG2, frame_size_in_bytes + padding + 0, R1_SP); + call_offset = __ call_RT(result, noreg, target); + break; + default: Unimplemented(); break; + } + OopMapSet* oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + restore_live_registers(sasm, result, noreg); + if (do_return) __ blr(); + return oop_maps; +} + + +OopMapSet* Runtime1::generate_patching(StubAssembler* sasm, address target) { + // Make a frame and preserve the caller's caller-save registers. + OopMap* oop_map = save_live_registers(sasm); + + // Call the runtime patching routine, returns non-zero if nmethod got deopted. + int call_offset = __ call_RT(noreg, noreg, target); + OopMapSet* oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + __ cmpdi(CCR0, R3_RET, 0); + + // Re-execute the patched instruction or, if the nmethod was deoptmized, + // return to the deoptimization handler entry that will cause re-execution + // of the current bytecode. + DeoptimizationBlob* deopt_blob = SharedRuntime::deopt_blob(); + assert(deopt_blob != NULL, "deoptimization blob must have been created"); + + // Return to the deoptimization handler entry for unpacking and rexecute. + // If we simply returned the we'd deopt as if any call we patched had just + // returned. + + restore_live_registers(sasm, noreg, noreg); + // Return if patching routine returned 0. + __ bclr(Assembler::bcondCRbiIs1, Assembler::bi0(CCR0, Assembler::equal), Assembler::bhintbhBCLRisReturn); + + address stub = deopt_blob->unpack_with_reexecution(); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ bctr(); + + return oop_maps; +} + +OopMapSet* Runtime1::generate_code_for(StubID id, StubAssembler* sasm) { + OopMapSet* oop_maps = NULL; + + // For better readability. + const bool must_gc_arguments = true; + const bool dont_gc_arguments = false; + + // Stub code & info for the different stubs. + switch (id) { + case forward_exception_id: + { + oop_maps = generate_handle_exception(id, sasm); + } + break; + + case new_instance_id: + case fast_new_instance_id: + case fast_new_instance_init_check_id: + { + if (id == new_instance_id) { + __ set_info("new_instance", dont_gc_arguments); + } else if (id == fast_new_instance_id) { + __ set_info("fast new_instance", dont_gc_arguments); + } else { + assert(id == fast_new_instance_init_check_id, "bad StubID"); + __ set_info("fast new_instance init check", dont_gc_arguments); + } + // We don't support eden allocation. +// if ((id == fast_new_instance_id || id == fast_new_instance_init_check_id) && +// UseTLAB && FastTLABRefill) { +// if (id == fast_new_instance_init_check_id) { +// // make sure the klass is initialized +// __ lbz(R0, in_bytes(InstanceKlass::init_state_offset()), R3_ARG1); +// __ cmpwi(CCR0, R0, InstanceKlass::fully_initialized); +// __ bne(CCR0, slow_path); +// } +//#ifdef ASSERT +// // assert object can be fast path allocated +// { +// Label ok, not_ok; +// __ lwz(R0, in_bytes(Klass::layout_helper_offset()), R3_ARG1); +// // make sure it's an instance (LH > 0) +// __ cmpwi(CCR0, R0, 0); +// __ ble(CCR0, not_ok); +// __ testbitdi(CCR0, R0, R0, Klass::_lh_instance_slow_path_bit); +// __ beq(CCR0, ok); +// +// __ bind(not_ok); +// __ stop("assert(can be fast path allocated)"); +// __ bind(ok); +// } +//#endif // ASSERT +// // We don't support eden allocation. +// __ bind(slow_path); +// } + oop_maps = generate_stub_call(sasm, R3_RET, CAST_FROM_FN_PTR(address, new_instance), R4_ARG2); + } + break; + + case counter_overflow_id: + // Bci and method are on stack. + oop_maps = stub_call_with_stack_parms(sasm, noreg, CAST_FROM_FN_PTR(address, counter_overflow), 2); + break; + + case new_type_array_id: + case new_object_array_id: + { + if (id == new_type_array_id) { + __ set_info("new_type_array", dont_gc_arguments); + } else { + __ set_info("new_object_array", dont_gc_arguments); + } + +#ifdef ASSERT + // Assert object type is really an array of the proper kind. + { + int tag = (id == new_type_array_id) ? Klass::_lh_array_tag_type_value : Klass::_lh_array_tag_obj_value; + Label ok; + __ lwz(R0, in_bytes(Klass::layout_helper_offset()), R4_ARG2); + __ srawi(R0, R0, Klass::_lh_array_tag_shift); + __ cmpwi(CCR0, R0, tag); + __ beq(CCR0, ok); + __ stop("assert(is an array klass)"); + __ should_not_reach_here(); + __ bind(ok); + } +#endif // ASSERT + + // We don't support eden allocation. + + if (id == new_type_array_id) { + oop_maps = generate_stub_call(sasm, R3_RET, CAST_FROM_FN_PTR(address, new_type_array), R4_ARG2, R5_ARG3); + } else { + oop_maps = generate_stub_call(sasm, R3_RET, CAST_FROM_FN_PTR(address, new_object_array), R4_ARG2, R5_ARG3); + } + } + break; + + case new_multi_array_id: + { + // R4: klass + // R5: rank + // R6: address of 1st dimension + __ set_info("new_multi_array", dont_gc_arguments); + oop_maps = generate_stub_call(sasm, R3_RET, CAST_FROM_FN_PTR(address, new_multi_array), R4_ARG2, R5_ARG3, R6_ARG4); + } + break; + + case register_finalizer_id: + { + __ set_info("register_finalizer", dont_gc_arguments); + // This code is called via rt_call. Hence, caller-save registers have been saved. + Register t = R11_scratch1; + + // Load the klass and check the has finalizer flag. + __ load_klass(t, R3_ARG1); + __ lwz(t, in_bytes(Klass::access_flags_offset()), t); + __ testbitdi(CCR0, R0, t, exact_log2(JVM_ACC_HAS_FINALIZER)); + // Return if has_finalizer bit == 0 (CR0.eq). + __ bclr(Assembler::bcondCRbiIs1, Assembler::bi0(CCR0, Assembler::equal), Assembler::bhintbhBCLRisReturn); + + __ mflr(R0); + __ std(R0, _abi(lr), R1_SP); + __ push_frame(frame::abi_reg_args_size, R0); // Empty dummy frame (no callee-save regs). + sasm->set_frame_size(frame::abi_reg_args_size / BytesPerWord); + OopMap* oop_map = new OopMap(frame::abi_reg_args_size / sizeof(jint), 0); + int call_offset = __ call_RT(noreg, noreg, + CAST_FROM_FN_PTR(address, SharedRuntime::register_finalizer), R3_ARG1); + oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + __ pop_frame(); + __ ld(R0, _abi(lr), R1_SP); + __ mtlr(R0); + __ blr(); + } + break; + + case throw_range_check_failed_id: + { + __ set_info("range_check_failed", dont_gc_arguments); // Arguments will be discarded. + __ std(R0, -8, R1_SP); // Pass index on stack. + oop_maps = generate_exception_throw_with_stack_parms(sasm, CAST_FROM_FN_PTR(address, throw_range_check_exception), 1); + } + break; + + case throw_index_exception_id: + { + __ set_info("index_range_check_failed", dont_gc_arguments); // Arguments will be discarded. + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_index_exception), true); + } + break; + + case throw_div0_exception_id: + { + __ set_info("throw_div0_exception", dont_gc_arguments); + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_div0_exception), false); + } + break; + + case throw_null_pointer_exception_id: + { + __ set_info("throw_null_pointer_exception", dont_gc_arguments); + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_null_pointer_exception), false); + } + break; + + case handle_exception_nofpu_id: + case handle_exception_id: + { + __ set_info("handle_exception", dont_gc_arguments); + oop_maps = generate_handle_exception(id, sasm); + } + break; + + case handle_exception_from_callee_id: + { + __ set_info("handle_exception_from_callee", dont_gc_arguments); + oop_maps = generate_handle_exception(id, sasm); + } + break; + + case unwind_exception_id: + { + const Register Rexception = R3 /*LIRGenerator::exceptionOopOpr()*/, + Rexception_pc = R4 /*LIRGenerator::exceptionPcOpr()*/, + Rexception_save = R31, Rcaller_sp = R30; + __ set_info("unwind_exception", dont_gc_arguments); + + __ ld(Rcaller_sp, 0, R1_SP); + __ push_frame_reg_args(0, R0); // dummy frame for C call + __ mr(Rexception_save, Rexception); // save over C call + __ ld(Rexception_pc, _abi(lr), Rcaller_sp); // return pc + __ call_VM_leaf(CAST_FROM_FN_PTR(address, SharedRuntime::exception_handler_for_return_address), R16_thread, Rexception_pc); + __ verify_not_null_oop(Rexception_save); + __ mtctr(R3_RET); + __ ld(Rexception_pc, _abi(lr), Rcaller_sp); // return pc + __ mr(R1_SP, Rcaller_sp); // Pop both frames at once. + __ mr(Rexception, Rexception_save); // restore + __ mtlr(Rexception_pc); + __ bctr(); + } + break; + + case throw_array_store_exception_id: + { + __ set_info("throw_array_store_exception", dont_gc_arguments); + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_array_store_exception), true); + } + break; + + case throw_class_cast_exception_id: + { + __ set_info("throw_class_cast_exception", dont_gc_arguments); + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_class_cast_exception), true); + } + break; + + case throw_incompatible_class_change_error_id: + { + __ set_info("throw_incompatible_class_cast_exception", dont_gc_arguments); + oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_incompatible_class_change_error), false); + } + break; + + case slow_subtype_check_id: + { // Support for uint StubRoutine::partial_subtype_check( Klass sub, Klass super ); + const Register sub_klass = R5, + super_klass = R4, + temp1_reg = R6, + temp2_reg = R0; + __ check_klass_subtype_slow_path(sub_klass, super_klass, temp1_reg, temp2_reg); // returns with CR0.eq if successful + __ crandc(CCR0, Assembler::equal, CCR0, Assembler::equal); // failed: CR0.ne + __ blr(); + } + break; + + case monitorenter_nofpu_id: + case monitorenter_id: + { + __ set_info("monitorenter", dont_gc_arguments); + + int save_fpu_registers = (id == monitorenter_id); + // Make a frame and preserve the caller's caller-save registers. + OopMap* oop_map = save_live_registers(sasm, save_fpu_registers); + + int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, monitorenter), R4_ARG2, R5_ARG3); + + oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + restore_live_registers(sasm, noreg, noreg, save_fpu_registers); + __ blr(); + } + break; + + case monitorexit_nofpu_id: + case monitorexit_id: + { + // note: Really a leaf routine but must setup last java sp + // => use call_RT for now (speed can be improved by + // doing last java sp setup manually). + __ set_info("monitorexit", dont_gc_arguments); + + int save_fpu_registers = (id == monitorexit_id); + // Make a frame and preserve the caller's caller-save registers. + OopMap* oop_map = save_live_registers(sasm, save_fpu_registers); + + int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, monitorexit), R4_ARG2); + + oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + restore_live_registers(sasm, noreg, noreg, save_fpu_registers); + __ blr(); + } + break; + + case deoptimize_id: + { + __ set_info("deoptimize", dont_gc_arguments); + __ std(R0, -8, R1_SP); // Pass trap_request on stack. + oop_maps = stub_call_with_stack_parms(sasm, noreg, CAST_FROM_FN_PTR(address, deoptimize), 1, /*do_return*/ false); + + DeoptimizationBlob* deopt_blob = SharedRuntime::deopt_blob(); + assert(deopt_blob != NULL, "deoptimization blob must have been created"); + address stub = deopt_blob->unpack_with_reexecution(); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ bctr(); + } + break; + + case access_field_patching_id: + { + __ set_info("access_field_patching", dont_gc_arguments); + oop_maps = generate_patching(sasm, CAST_FROM_FN_PTR(address, access_field_patching)); + } + break; + + case load_klass_patching_id: + { + __ set_info("load_klass_patching", dont_gc_arguments); + oop_maps = generate_patching(sasm, CAST_FROM_FN_PTR(address, move_klass_patching)); + } + break; + + case load_mirror_patching_id: + { + __ set_info("load_mirror_patching", dont_gc_arguments); + oop_maps = generate_patching(sasm, CAST_FROM_FN_PTR(address, move_mirror_patching)); + } + break; + + case load_appendix_patching_id: + { + __ set_info("load_appendix_patching", dont_gc_arguments); + oop_maps = generate_patching(sasm, CAST_FROM_FN_PTR(address, move_appendix_patching)); + } + break; + + case dtrace_object_alloc_id: + { // O0: object + __ unimplemented("stub dtrace_object_alloc_id"); + __ set_info("dtrace_object_alloc", dont_gc_arguments); +// // We can't gc here so skip the oopmap but make sure that all +// // the live registers get saved. +// save_live_registers(sasm); +// +// __ save_thread(L7_thread_cache); +// __ call(CAST_FROM_FN_PTR(address, SharedRuntime::dtrace_object_alloc), +// relocInfo::runtime_call_type); +// __ delayed()->mov(I0, O0); +// __ restore_thread(L7_thread_cache); +// +// restore_live_registers(sasm); +// __ ret(); +// __ delayed()->restore(); + } + break; + +#if INCLUDE_ALL_GCS + case g1_pre_barrier_slow_id: + { + BarrierSet* bs = Universe::heap()->barrier_set(); + if (bs->kind() != BarrierSet::G1SATBCTLogging) { + goto unimplemented_entry; + } + + __ set_info("g1_pre_barrier_slow_id", dont_gc_arguments); + + // Using stack slots: pre_val (pre-pushed), spill tmp, spill tmp2. + const int stack_slots = 3; + Register pre_val = R0; // previous value of memory + Register tmp = R14; + Register tmp2 = R15; + + Label refill, restart; + int satb_q_index_byte_offset = + in_bytes(JavaThread::satb_mark_queue_offset() + + SATBMarkQueue::byte_offset_of_index()); + int satb_q_buf_byte_offset = + in_bytes(JavaThread::satb_mark_queue_offset() + + SATBMarkQueue::byte_offset_of_buf()); + + // Spill + __ std(tmp, -16, R1_SP); + __ std(tmp2, -24, R1_SP); + + __ bind(restart); + // Load the index into the SATB buffer. SATBMarkQueue::_index is a + // size_t so ld_ptr is appropriate. + __ ld(tmp, satb_q_index_byte_offset, R16_thread); + + // index == 0? + __ cmpdi(CCR0, tmp, 0); + __ beq(CCR0, refill); + + __ ld(tmp2, satb_q_buf_byte_offset, R16_thread); + __ ld(pre_val, -8, R1_SP); // Load from stack. + __ addi(tmp, tmp, -oopSize); + + __ std(tmp, satb_q_index_byte_offset, R16_thread); + __ stdx(pre_val, tmp2, tmp); // [_buf + index] := + + // Restore temp registers and return-from-leaf. + __ ld(tmp2, -24, R1_SP); + __ ld(tmp, -16, R1_SP); + __ blr(); + + __ bind(refill); + const int nbytes_save = (MacroAssembler::num_volatile_regs + stack_slots) * BytesPerWord; + __ save_volatile_gprs(R1_SP, -nbytes_save); // except R0 + __ mflr(R0); + __ std(R0, _abi(lr), R1_SP); + __ push_frame_reg_args(nbytes_save, R0); // dummy frame for C call + __ call_VM_leaf(CAST_FROM_FN_PTR(address, SATBMarkQueueSet::handle_zero_index_for_thread), R16_thread); + __ pop_frame(); + __ ld(R0, _abi(lr), R1_SP); + __ mtlr(R0); + __ restore_volatile_gprs(R1_SP, -nbytes_save); // except R0 + __ b(restart); + } + break; + + case g1_post_barrier_slow_id: + { + BarrierSet* bs = Universe::heap()->barrier_set(); + if (bs->kind() != BarrierSet::G1SATBCTLogging) { + goto unimplemented_entry; + } + + __ set_info("g1_post_barrier_slow_id", dont_gc_arguments); + + // Using stack slots: spill addr, spill tmp2 + const int stack_slots = 2; + Register tmp = R0; + Register addr = R14; + Register tmp2 = R15; + jbyte* byte_map_base = ((CardTableModRefBS*)bs)->byte_map_base; + + Label restart, refill, ret; + + // Spill + __ std(addr, -8, R1_SP); + __ std(tmp2, -16, R1_SP); + + __ srdi(addr, R0, CardTableModRefBS::card_shift); // Addr is passed in R0. + __ load_const_optimized(/*cardtable*/ tmp2, byte_map_base, tmp); + __ add(addr, tmp2, addr); + __ lbz(tmp, 0, addr); // tmp := [addr + cardtable] + + // Return if young card. + __ cmpwi(CCR0, tmp, G1SATBCardTableModRefBS::g1_young_card_val()); + __ beq(CCR0, ret); + + // Return if sequential consistent value is already dirty. + __ membar(Assembler::StoreLoad); + __ lbz(tmp, 0, addr); // tmp := [addr + cardtable] + + __ cmpwi(CCR0, tmp, G1SATBCardTableModRefBS::dirty_card_val()); + __ beq(CCR0, ret); + + // Not dirty. + + // First, dirty it. + __ li(tmp, G1SATBCardTableModRefBS::dirty_card_val()); + __ stb(tmp, 0, addr); + + int dirty_card_q_index_byte_offset = + in_bytes(JavaThread::dirty_card_queue_offset() + + DirtyCardQueue::byte_offset_of_index()); + int dirty_card_q_buf_byte_offset = + in_bytes(JavaThread::dirty_card_queue_offset() + + DirtyCardQueue::byte_offset_of_buf()); + + __ bind(restart); + + // Get the index into the update buffer. DirtyCardQueue::_index is + // a size_t so ld_ptr is appropriate here. + __ ld(tmp2, dirty_card_q_index_byte_offset, R16_thread); + + // index == 0? + __ cmpdi(CCR0, tmp2, 0); + __ beq(CCR0, refill); + + __ ld(tmp, dirty_card_q_buf_byte_offset, R16_thread); + __ addi(tmp2, tmp2, -oopSize); + + __ std(tmp2, dirty_card_q_index_byte_offset, R16_thread); + __ add(tmp2, tmp, tmp2); + __ std(addr, 0, tmp2); // [_buf + index] := + + // Restore temp registers and return-from-leaf. + __ bind(ret); + __ ld(tmp2, -16, R1_SP); + __ ld(addr, -8, R1_SP); + __ blr(); + + __ bind(refill); + const int nbytes_save = (MacroAssembler::num_volatile_regs + stack_slots) * BytesPerWord; + __ save_volatile_gprs(R1_SP, -nbytes_save); // except R0 + __ mflr(R0); + __ std(R0, _abi(lr), R1_SP); + __ push_frame_reg_args(nbytes_save, R0); // dummy frame for C call + __ call_VM_leaf(CAST_FROM_FN_PTR(address, DirtyCardQueueSet::handle_zero_index_for_thread), R16_thread); + __ pop_frame(); + __ ld(R0, _abi(lr), R1_SP); + __ mtlr(R0); + __ restore_volatile_gprs(R1_SP, -nbytes_save); // except R0 + __ b(restart); + } + break; +#endif // INCLUDE_ALL_GCS + + case predicate_failed_trap_id: + { + __ set_info("predicate_failed_trap", dont_gc_arguments); + OopMap* oop_map = save_live_registers(sasm); + + int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, predicate_failed_trap)); + + oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + + DeoptimizationBlob* deopt_blob = SharedRuntime::deopt_blob(); + assert(deopt_blob != NULL, "deoptimization blob must have been created"); + restore_live_registers(sasm, noreg, noreg); + + address stub = deopt_blob->unpack_with_reexecution(); + //__ load_const_optimized(R0, stub); + __ add_const_optimized(R0, R29_TOC, MacroAssembler::offset_to_global_toc(stub)); + __ mtctr(R0); + __ bctr(); + } + break; + + default: + unimplemented_entry: + { + __ set_info("unimplemented entry", dont_gc_arguments); + __ mflr(R0); + __ std(R0, _abi(lr), R1_SP); + __ push_frame(frame::abi_reg_args_size, R0); // empty dummy frame + sasm->set_frame_size(frame::abi_reg_args_size / BytesPerWord); + OopMap* oop_map = new OopMap(frame::abi_reg_args_size / sizeof(jint), 0); + + __ load_const_optimized(R4_ARG2, (int)id); + int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, unimplemented_entry), R4_ARG2); + + oop_maps = new OopMapSet(); + oop_maps->add_gc_map(call_offset, oop_map); + __ should_not_reach_here(); + } + break; + } + return oop_maps; +} + + +OopMapSet* Runtime1::generate_handle_exception(StubID id, StubAssembler* sasm) { + __ block_comment("generate_handle_exception"); + + // Save registers, if required. + OopMapSet* oop_maps = new OopMapSet(); + OopMap* oop_map = NULL; + const Register Rexception = R3 /*LIRGenerator::exceptionOopOpr()*/, + Rexception_pc = R4 /*LIRGenerator::exceptionPcOpr()*/; + + switch (id) { + case forward_exception_id: + // We're handling an exception in the context of a compiled frame. + // The registers have been saved in the standard places. Perform + // an exception lookup in the caller and dispatch to the handler + // if found. Otherwise unwind and dispatch to the callers + // exception handler. + oop_map = generate_oop_map(sasm, true); + // Transfer the pending exception to the exception_oop. + // Also load the PC which is typically at SP + frame_size_in_bytes + _abi(lr), + // but we support additional slots in the frame for parameter passing. + __ ld(Rexception_pc, 0, R1_SP); + __ ld(Rexception, in_bytes(JavaThread::pending_exception_offset()), R16_thread); + __ li(R0, 0); + __ ld(Rexception_pc, _abi(lr), Rexception_pc); + __ std(R0, in_bytes(JavaThread::pending_exception_offset()), R16_thread); + break; + case handle_exception_nofpu_id: + case handle_exception_id: + // At this point all registers MAY be live. + oop_map = save_live_registers(sasm, id != handle_exception_nofpu_id, Rexception_pc); + break; + case handle_exception_from_callee_id: + // At this point all registers except exception oop and exception pc are dead. + oop_map = new OopMap(frame_size_in_bytes / sizeof(jint), 0); + sasm->set_frame_size(frame_size_in_bytes / BytesPerWord); + __ std(Rexception_pc, _abi(lr), R1_SP); + __ push_frame(frame_size_in_bytes, R0); + break; + default: ShouldNotReachHere(); + } + + __ verify_not_null_oop(Rexception); + +#ifdef ASSERT + // Check that fields in JavaThread for exception oop and issuing pc are + // empty before writing to them. + __ ld(R0, in_bytes(JavaThread::exception_oop_offset()), R16_thread); + __ cmpdi(CCR0, R0, 0); + __ asm_assert_eq("exception oop already set", 0x963); + __ ld(R0, in_bytes(JavaThread::exception_pc_offset() ), R16_thread); + __ cmpdi(CCR0, R0, 0); + __ asm_assert_eq("exception pc already set", 0x962); +#endif + + // Save the exception and issuing pc in the thread. + __ std(Rexception, in_bytes(JavaThread::exception_oop_offset()), R16_thread); + __ std(Rexception_pc, in_bytes(JavaThread::exception_pc_offset() ), R16_thread); + + int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, exception_handler_for_pc)); + oop_maps->add_gc_map(call_offset, oop_map); + + __ mtctr(R3_RET); + + // Note: if nmethod has been deoptimized then regardless of + // whether it had a handler or not we will deoptimize + // by entering the deopt blob with a pending exception. + + // Restore the registers that were saved at the beginning, remove + // the frame and jump to the exception handler. + switch (id) { + case forward_exception_id: + case handle_exception_nofpu_id: + case handle_exception_id: + restore_live_registers(sasm, noreg, noreg, id != handle_exception_nofpu_id); + __ bctr(); + break; + case handle_exception_from_callee_id: { + __ pop_frame(); + __ ld(Rexception_pc, _abi(lr), R1_SP); + __ mtlr(Rexception_pc); + __ bctr(); + break; + } + default: ShouldNotReachHere(); + } + + return oop_maps; +} + +const char *Runtime1::pd_name_for_address(address entry) { + return ""; +} + +#undef __ diff --git a/hotspot/src/cpu/ppc/vm/c1_globals_ppc.hpp b/hotspot/src/cpu/ppc/vm/c1_globals_ppc.hpp new file mode 100644 index 00000000000..234248e387e --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/c1_globals_ppc.hpp @@ -0,0 +1,68 @@ +/* + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_C1_GLOBALS_PPC_HPP +#define CPU_PPC_VM_C1_GLOBALS_PPC_HPP + +#include "utilities/globalDefinitions.hpp" +#include "utilities/macros.hpp" + +// Sets the default values for platform dependent flags used by the client compiler. +// (see c1_globals.hpp) + +#ifndef TIERED +define_pd_global(bool, BackgroundCompilation, true ); +define_pd_global(bool, CICompileOSR, true ); +define_pd_global(bool, InlineIntrinsics, true ); +define_pd_global(bool, PreferInterpreterNativeStubs, false); +define_pd_global(bool, ProfileTraps, false); +define_pd_global(bool, UseOnStackReplacement, true ); +define_pd_global(bool, TieredCompilation, false); +define_pd_global(intx, CompileThreshold, 1000 ); + +define_pd_global(intx, OnStackReplacePercentage, 1400 ); +define_pd_global(bool, UseTLAB, true ); +define_pd_global(bool, ProfileInterpreter, false); +define_pd_global(intx, FreqInlineSize, 325 ); +define_pd_global(bool, ResizeTLAB, true ); +define_pd_global(intx, ReservedCodeCacheSize, 32*M ); +define_pd_global(intx, CodeCacheExpansionSize, 32*K ); +define_pd_global(uintx,CodeCacheMinBlockLength, 1); +define_pd_global(uintx,MetaspaceSize, 12*M ); +define_pd_global(bool, NeverActAsServerClassMachine, true ); +define_pd_global(intx, NewSizeThreadIncrease, 16*K ); +define_pd_global(uint64_t,MaxRAM, 1ULL*G); +define_pd_global(intx, InitialCodeCacheSize, 160*K); +#endif // !TIERED + +define_pd_global(bool, UseTypeProfile, false); +define_pd_global(bool, RoundFPResults, false); + +define_pd_global(bool, LIRFillDelaySlots, false); +define_pd_global(bool, OptimizeSinglePrecision, false); +define_pd_global(bool, CSEArrayLength, true ); +define_pd_global(bool, TwoOperandLIRForm, false); + +#endif // CPU_PPC_VM_C1_GLOBALS_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/c2_globals_ppc.hpp b/hotspot/src/cpu/ppc/vm/c2_globals_ppc.hpp index 20174522381..9934d1f0143 100644 --- a/hotspot/src/cpu/ppc/vm/c2_globals_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/c2_globals_ppc.hpp @@ -39,7 +39,7 @@ define_pd_global(bool, PreferInterpreterNativeStubs, false); define_pd_global(bool, ProfileTraps, true); define_pd_global(bool, UseOnStackReplacement, true); define_pd_global(bool, ProfileInterpreter, true); -define_pd_global(bool, TieredCompilation, false); +define_pd_global(bool, TieredCompilation, true); define_pd_global(intx, CompileThreshold, 10000); define_pd_global(intx, OnStackReplacePercentage, 140); diff --git a/hotspot/src/cpu/ppc/vm/c2_init_ppc.cpp b/hotspot/src/cpu/ppc/vm/c2_init_ppc.cpp index 94b59db4d7e..a9fe522671c 100644 --- a/hotspot/src/cpu/ppc/vm/c2_init_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/c2_init_ppc.cpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2013 SAP AG. All rights reserved. + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -45,4 +45,14 @@ void Compile::pd_compiler2_init() { FLAG_SET_ERGO(bool, InsertEndGroupPPC64, true); } } + + if (!VM_Version::has_isel() && FLAG_IS_DEFAULT(ConditionalMoveLimit)) { + FLAG_SET_ERGO(intx, ConditionalMoveLimit, 0); + } + + if (OptimizeFill) { + warning("OptimizeFill is not supported on this CPU."); + FLAG_SET_DEFAULT(OptimizeFill, false); + } + } diff --git a/hotspot/src/cpu/ppc/vm/compiledIC_ppc.cpp b/hotspot/src/cpu/ppc/vm/compiledIC_ppc.cpp index 41b5125f06e..197d3069817 100644 --- a/hotspot/src/cpu/ppc/vm/compiledIC_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/compiledIC_ppc.cpp @@ -1,5 +1,6 @@ /* - * Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -129,13 +130,20 @@ address CompiledStaticCall::emit_to_interp_stub(CodeBuffer &cbuf, address mark/* // - call __ calculate_address_from_global_toc(reg_scratch, __ method_toc()); AddressLiteral ic = __ allocate_metadata_address((Metadata *)NULL); - __ load_const_from_method_toc(as_Register(Matcher::inline_cache_reg_encode()), ic, reg_scratch); + bool success = __ load_const_from_method_toc(as_Register(Matcher::inline_cache_reg_encode()), + ic, reg_scratch, /*fixed_size*/ true); + if (!success) { + return NULL; // CodeCache is full + } if (ReoptimizeCallSequences) { __ b64_patchable((address)-1, relocInfo::none); } else { AddressLiteral a((address)-1); - __ load_const_from_method_toc(reg_scratch, a, reg_scratch); + success = __ load_const_from_method_toc(reg_scratch, a, reg_scratch, /*fixed_size*/ true); + if (!success) { + return NULL; // CodeCache is full + } __ mtctr(reg_scratch); __ bctr(); } @@ -153,6 +161,7 @@ address CompiledStaticCall::emit_to_interp_stub(CodeBuffer &cbuf, address mark/* return stub; #else ShouldNotReachHere(); + return NULL; #endif } #undef __ diff --git a/hotspot/src/cpu/ppc/vm/frame_ppc.cpp b/hotspot/src/cpu/ppc/vm/frame_ppc.cpp index 69646f74c77..29921834ac1 100644 --- a/hotspot/src/cpu/ppc/vm/frame_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/frame_ppc.cpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2014 SAP AG. All rights reserved. + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -236,39 +236,6 @@ void frame::describe_pd(FrameValues& values, int frame_no) { } #endif -void frame::adjust_unextended_sp() { - // If we are returning to a compiled MethodHandle call site, the - // saved_fp will in fact be a saved value of the unextended SP. The - // simplest way to tell whether we are returning to such a call site - // is as follows: - - if (is_compiled_frame() && false /*is_at_mh_callsite()*/) { // TODO PPC port - // If the sender PC is a deoptimization point, get the original - // PC. For MethodHandle call site the unextended_sp is stored in - // saved_fp. - _unextended_sp = _fp - _cb->frame_size(); - -#ifdef ASSERT - nmethod *sender_nm = _cb->as_nmethod_or_null(); - assert(sender_nm && *_sp == *_unextended_sp, "backlink changed"); - - intptr_t* sp = _unextended_sp; // check if stack can be walked from here - for (int x = 0; x < 5; ++x) { // check up to a couple of backlinks - intptr_t* prev_sp = *(intptr_t**)sp; - if (prev_sp == 0) break; // end of stack - assert(prev_sp>sp, "broken stack"); - sp = prev_sp; - } - - if (sender_nm->is_deopt_mh_entry(_pc)) { // checks for deoptimization - address original_pc = sender_nm->get_original_pc(this); - assert(sender_nm->insts_contains(original_pc), "original PC must be in nmethod"); - assert(sender_nm->is_method_handle_return(original_pc), "must be"); - } -#endif - } -} - intptr_t *frame::initial_deoptimization_info() { // unused... but returns fp() to minimize changes introduced by 7087445 return fp(); diff --git a/hotspot/src/cpu/ppc/vm/frame_ppc.hpp b/hotspot/src/cpu/ppc/vm/frame_ppc.hpp index 8a1ff5c63b3..3b90da157a5 100644 --- a/hotspot/src/cpu/ppc/vm/frame_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/frame_ppc.hpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2014 SAP AG. All rights reserved. + * Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -373,7 +373,6 @@ // The frame's stack pointer before it has been extended by a c2i adapter; // needed by deoptimization intptr_t* _unextended_sp; - void adjust_unextended_sp(); public: diff --git a/hotspot/src/cpu/ppc/vm/frame_ppc.inline.hpp b/hotspot/src/cpu/ppc/vm/frame_ppc.inline.hpp index 097ac0da6a1..2c90cd2a6a1 100644 --- a/hotspot/src/cpu/ppc/vm/frame_ppc.inline.hpp +++ b/hotspot/src/cpu/ppc/vm/frame_ppc.inline.hpp @@ -39,9 +39,6 @@ inline void frame::find_codeblob_and_set_pc_and_deopt_state(address pc) { _pc = pc; // Must be set for get_deopt_original_pc() _fp = (intptr_t*)own_abi()->callers_sp; - // Use _fp - frame_size, needs to be done between _cb and _pc initialization - // and get_deopt_original_pc. - adjust_unextended_sp(); address original_pc = nmethod::get_deopt_original_pc(this); if (original_pc != NULL) { diff --git a/hotspot/src/cpu/ppc/vm/globalDefinitions_ppc.hpp b/hotspot/src/cpu/ppc/vm/globalDefinitions_ppc.hpp index fabffc8c3cf..f7aa82d6cde 100644 --- a/hotspot/src/cpu/ppc/vm/globalDefinitions_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/globalDefinitions_ppc.hpp @@ -35,11 +35,18 @@ const int BytesPerInstWord = 4; const int StackAlignmentInBytes = 16; +// Indicates whether the C calling conventions require that +// 32-bit integer argument values are extended to 64 bits. +const bool CCallingConventionRequiresIntsAsLongs = true; + #define SUPPORTS_NATIVE_CX8 // The PPC CPUs are NOT multiple-copy-atomic. #define CPU_NOT_MULTIPLE_COPY_ATOMIC +// The expected size in bytes of a cache line, used to pad data structures. +#define DEFAULT_CACHE_LINE_SIZE 128 + #if defined(COMPILER2) && defined(AIX) // Include Transactional Memory lock eliding optimization #define INCLUDE_RTM_OPT 1 diff --git a/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.cpp b/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.cpp index 80350fe4c5d..9c7128da0b4 100644 --- a/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.cpp +++ b/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.cpp @@ -87,9 +87,9 @@ void InterpreterMacroAssembler::dispatch_prolog(TosState state, int bcp_incr) { // own dispatch. The dispatch address in R24_dispatch_addr is used for the // dispatch. void InterpreterMacroAssembler::dispatch_epilog(TosState state, int bcp_incr) { + if (bcp_incr) { addi(R14_bcp, R14_bcp, bcp_incr); } mtctr(R24_dispatch_addr); - addi(R14_bcp, R14_bcp, bcp_incr); - bctr(); + bcctr(bcondAlways, 0, bhintbhBCCTRisNotPredictable); } void InterpreterMacroAssembler::check_and_handle_popframe(Register scratch_reg) { @@ -207,9 +207,6 @@ void InterpreterMacroAssembler::dispatch_Lbyte_code(TosState state, Register byt unimplemented("dispatch_Lbyte_code: verify"); // See Sparc Implementation to implement this } -#ifdef FAST_DISPATCH - unimplemented("dispatch_Lbyte_code FAST_DISPATCH"); -#else assert_different_registers(bytecode, R11_scratch1); // Calc dispatch table address. @@ -220,8 +217,7 @@ void InterpreterMacroAssembler::dispatch_Lbyte_code(TosState state, Register byt // Jump off! mtctr(R11_scratch1); - bctr(); -#endif + bcctr(bcondAlways, 0, bhintbhBCCTRisNotPredictable); } void InterpreterMacroAssembler::load_receiver(Register Rparam_count, Register Rrecv_dst) { @@ -544,8 +540,8 @@ void InterpreterMacroAssembler::index_check_without_pop(Register Rarray, Registe sldi(RsxtIndex, RsxtIndex, index_shift); blt(CCR0, LnotOOR); // Index should be in R17_tos, array should be in R4_ARG2. - mr(R17_tos, Rindex); - mr(R4_ARG2, Rarray); + mr_if_needed(R17_tos, Rindex); + mr_if_needed(R4_ARG2, Rarray); load_dispatch_table(Rtmp, (address*)Interpreter::_throw_ArrayIndexOutOfBoundsException_entry); mtctr(Rtmp); bctr(); @@ -842,7 +838,6 @@ void InterpreterMacroAssembler::lock_object(Register monitor, Register object) { // Must fence, otherwise, preceding store(s) may float below cmpxchg. // CmpxchgX sets CCR0 to cmpX(current, displaced). - fence(); // TODO: replace by MacroAssembler::MemBarRel | MacroAssembler::MemBarAcq ? cmpxchgd(/*flag=*/CCR0, /*current_value=*/current_header, /*compare_value=*/displaced_header, /*exchange_value=*/monitor, @@ -850,7 +845,8 @@ void InterpreterMacroAssembler::lock_object(Register monitor, Register object) { MacroAssembler::MemBarRel | MacroAssembler::MemBarAcq, MacroAssembler::cmpxchgx_hint_acquire_lock(), noreg, - &cas_failed); + &cas_failed, + /*check without membar and ldarx first*/true); // If the compare-and-exchange succeeded, then we found an unlocked // object and we have now locked it. @@ -868,9 +864,7 @@ void InterpreterMacroAssembler::lock_object(Register monitor, Register object) { sub(current_header, current_header, R1_SP); assert(os::vm_page_size() > 0xfff, "page size too small - change the constant"); - load_const_optimized(tmp, - (address) (~(os::vm_page_size()-1) | - markOopDesc::lock_mask_in_place)); + load_const_optimized(tmp, ~(os::vm_page_size()-1) | markOopDesc::lock_mask_in_place); and_(R0/*==0?*/, current_header, tmp); // If condition is true we are done and hence we can store 0 in the displaced @@ -1106,6 +1100,7 @@ void InterpreterMacroAssembler::verify_method_data_pointer() { } void InterpreterMacroAssembler::test_invocation_counter_for_mdp(Register invocation_count, + Register method_counters, Register Rscratch, Label &profile_continue) { assert(ProfileInterpreter, "must be profiling interpreter"); @@ -1114,12 +1109,11 @@ void InterpreterMacroAssembler::test_invocation_counter_for_mdp(Register invocat Label done; // If no method data exists, and the counter is high enough, make one. - int ipl_offs = load_const_optimized(Rscratch, &InvocationCounter::InterpreterProfileLimit, R0, true); - lwz(Rscratch, ipl_offs, Rscratch); + lwz(Rscratch, in_bytes(MethodCounters::interpreter_profile_limit_offset()), method_counters); cmpdi(CCR0, R28_mdx, 0); // Test to see if we should create a method data oop. - cmpd(CCR1, Rscratch /* InterpreterProfileLimit */, invocation_count); + cmpd(CCR1, Rscratch, invocation_count); bne(CCR0, done); bge(CCR1, profile_continue); @@ -1132,15 +1126,15 @@ void InterpreterMacroAssembler::test_invocation_counter_for_mdp(Register invocat bind(done); } -void InterpreterMacroAssembler::test_backedge_count_for_osr(Register backedge_count, Register branch_bcp, Register Rtmp) { - assert_different_registers(backedge_count, Rtmp, branch_bcp); +void InterpreterMacroAssembler::test_backedge_count_for_osr(Register backedge_count, Register method_counters, + Register target_bcp, Register disp, Register Rtmp) { + assert_different_registers(backedge_count, target_bcp, disp, Rtmp, R4_ARG2); assert(UseOnStackReplacement,"Must UseOnStackReplacement to test_backedge_count_for_osr"); Label did_not_overflow; Label overflow_with_error; - int ibbl_offs = load_const_optimized(Rtmp, &InvocationCounter::InterpreterBackwardBranchLimit, R0, true); - lwz(Rtmp, ibbl_offs, Rtmp); + lwz(Rtmp, in_bytes(MethodCounters::interpreter_backward_branch_limit_offset()), method_counters); cmpw(CCR0, backedge_count, Rtmp); blt(CCR0, did_not_overflow); @@ -1152,17 +1146,15 @@ void InterpreterMacroAssembler::test_backedge_count_for_osr(Register backedge_co // the overflow function is called only once every overflow_frequency. if (ProfileInterpreter) { const int overflow_frequency = 1024; - li(Rtmp, overflow_frequency-1); - andr(Rtmp, Rtmp, backedge_count); - cmpwi(CCR0, Rtmp, 0); + andi_(Rtmp, backedge_count, overflow_frequency-1); bne(CCR0, did_not_overflow); } // Overflow in loop, pass branch bytecode. - call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::frequency_counter_overflow), branch_bcp, true); + subf(R4_ARG2, disp, target_bcp); // Compute branch bytecode (previous bcp). + call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::frequency_counter_overflow), R4_ARG2, true); // Was an OSR adapter generated? - // O0 = osr nmethod cmpdi(CCR0, R3_RET, 0); beq(CCR0, overflow_with_error); @@ -1323,7 +1315,7 @@ void InterpreterMacroAssembler::increment_backedge_counter(const Register Rcount assert_different_registers(Rdst, Rtmp1); const Register invocation_counter = Rtmp1; const Register counter = Rdst; - // TODO ppc port assert(4 == InvocationCounter::sz_counter(), "unexpected field size."); + // TODO: PPC port: assert(4 == InvocationCounter::sz_counter(), "unexpected field size."); // Load backedge counter. lwz(counter, in_bytes(MethodCounters::backedge_counter_offset()) + @@ -1336,8 +1328,7 @@ void InterpreterMacroAssembler::increment_backedge_counter(const Register Rcount addi(counter, counter, InvocationCounter::count_increment); // Mask the invocation counter. - li(Rscratch, InvocationCounter::count_mask_value); - andr(invocation_counter, invocation_counter, Rscratch); + andi(invocation_counter, invocation_counter, InvocationCounter::count_mask_value); // Store new counter value. stw(counter, in_bytes(MethodCounters::backedge_counter_offset()) + @@ -1822,15 +1813,13 @@ void InterpreterMacroAssembler::profile_return_type(Register ret, Register tmp1, test_method_data_pointer(profile_continue); if (MethodData::profile_return_jsr292_only()) { - assert(Method::intrinsic_id_size_in_bytes() == 2, "assuming Method::_intrinsic_id is u2"); - // If we don't profile all invoke bytecodes we must make sure // it's a bytecode we indeed profile. We can't go back to the // begining of the ProfileData we intend to update to check its // type because we're right after it and we don't known its // length. lbz(tmp1, 0, R14_bcp); - lhz(tmp2, Method::intrinsic_id_offset_in_bytes(), R19_method); + lbz(tmp2, Method::intrinsic_id_offset_in_bytes(), R19_method); cmpwi(CCR0, tmp1, Bytecodes::_invokedynamic); cmpwi(CCR1, tmp1, Bytecodes::_invokehandle); cror(CCR0, Assembler::equal, CCR1, Assembler::equal); @@ -2224,9 +2213,7 @@ void InterpreterMacroAssembler::increment_invocation_counter(Register Rcounters, // Load the backedge counter. lwz(backedge_count, be_counter_offset, Rcounters); // is unsigned int // Mask the backedge counter. - Register tmp = invocation_count; - li(tmp, InvocationCounter::count_mask_value); - andr(backedge_count, tmp, backedge_count); // Cannot use andi, need sign extension of count_mask_value. + andi(backedge_count, backedge_count, InvocationCounter::count_mask_value); // Load the invocation counter. lwz(invocation_count, inv_counter_offset, Rcounters); // is unsigned int @@ -2282,7 +2269,7 @@ void InterpreterMacroAssembler::verify_oop_or_return_address(Register reg, Regis bne(CCR0, test); address fd = CAST_FROM_FN_PTR(address, verify_return_address); - const int nbytes_save = 11*8; // volatile gprs except R0 + const int nbytes_save = MacroAssembler::num_volatile_regs * 8; save_volatile_gprs(R1_SP, -nbytes_save); // except R0 save_LR_CR(Rtmp); // Save in old frame. push_frame_reg_args(nbytes_save, Rtmp); diff --git a/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.hpp b/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.hpp index 7fd60ead67d..6c88cede5b8 100644 --- a/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.hpp +++ b/hotspot/src/cpu/ppc/vm/interp_masm_ppc_64.hpp @@ -195,7 +195,7 @@ class InterpreterMacroAssembler: public MacroAssembler { void restore_interpreter_state(Register scratch, bool bcp_and_mdx_only = false); void increment_backedge_counter(const Register Rcounters, Register Rtmp, Register Rtmp2, Register Rscratch); - void test_backedge_count_for_osr(Register backedge_count, Register branch_bcp, Register Rtmp); + void test_backedge_count_for_osr(Register backedge_count, Register method_counters, Register target_bcp, Register disp, Register Rtmp); void record_static_call_in_profile(Register Rentry, Register Rtmp); void record_receiver_call_in_profile(Register Rklass, Register Rentry, Register Rtmp); @@ -211,7 +211,7 @@ class InterpreterMacroAssembler: public MacroAssembler { void set_method_data_pointer_for_bcp(); void test_method_data_pointer(Label& zero_continue); void verify_method_data_pointer(); - void test_invocation_counter_for_mdp(Register invocation_count, Register Rscratch, Label &profile_continue); + void test_invocation_counter_for_mdp(Register invocation_count, Register method_counters, Register Rscratch, Label &profile_continue); void set_mdp_data_at(int constant, Register value); diff --git a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp index 87b16e0e572..fae424b569e 100644 --- a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.cpp @@ -30,6 +30,7 @@ #include "gc/shared/collectedHeap.inline.hpp" #include "interpreter/interpreter.hpp" #include "memory/resourceArea.hpp" +#include "nativeInst_ppc.hpp" #include "prims/methodHandles.hpp" #include "runtime/biasedLocking.hpp" #include "runtime/icache.hpp" @@ -114,7 +115,7 @@ void MacroAssembler::calculate_address_from_global_toc(Register dst, address add } if (hi16) { - addis(dst, R29, MacroAssembler::largeoffset_si16_si16_hi(offset)); + addis(dst, R29_TOC, MacroAssembler::largeoffset_si16_si16_hi(offset)); } if (lo16) { if (add_relocation) { @@ -256,7 +257,9 @@ narrowOop MacroAssembler::get_narrow_oop(address a, address bound) { } #endif // _LP64 -void MacroAssembler::load_const_from_method_toc(Register dst, AddressLiteral& a, Register toc) { +// Returns true if successful. +bool MacroAssembler::load_const_from_method_toc(Register dst, AddressLiteral& a, + Register toc, bool fixed_size) { int toc_offset = 0; // Use RelocationHolder::none for the constant pool entry, otherwise // we will end up with a failing NativeCall::verify(x) where x is @@ -264,11 +267,13 @@ void MacroAssembler::load_const_from_method_toc(Register dst, AddressLiteral& a, // FIXME: We should insert relocation information for oops at the constant // pool entries instead of inserting it at the loads; patching of a constant // pool entry should be less expensive. - address oop_address = address_constant((address)a.value(), RelocationHolder::none); + address const_address = address_constant((address)a.value(), RelocationHolder::none); + if (const_address == NULL) { return false; } // allocation failure // Relocate at the pc of the load. relocate(a.rspec()); - toc_offset = (int)(oop_address - code()->consts()->start()); - ld_largeoffset_unchecked(dst, toc_offset, toc, true); + toc_offset = (int)(const_address - code()->consts()->start()); + ld_largeoffset_unchecked(dst, toc_offset, toc, fixed_size); + return true; } bool MacroAssembler::is_load_const_from_method_toc_at(address a) { @@ -446,6 +451,15 @@ void MacroAssembler::bc_far(int boint, int biint, Label& dest, int optimize) { assert(dest.is_bound() || target_pc == b_pc, "postcondition"); } +// 1 or 2 instructions +void MacroAssembler::bc_far_optimized(int boint, int biint, Label& dest) { + if (dest.is_bound() && is_within_range_of_bcxx(target(dest), pc())) { + bc(boint, biint, dest); + } else { + bc_far(boint, biint, dest, MacroAssembler::bc_far_optimize_on_relocate); + } +} + bool MacroAssembler::is_bc_far_at(address instruction_addr) { return is_bc_far_variant1_at(instruction_addr) || is_bc_far_variant2_at(instruction_addr) || @@ -496,7 +510,7 @@ void MacroAssembler::set_dest_of_bc_far_at(address instruction_addr, address des // variant 1, the 1st instruction contains the destination address: // // bcxx DEST - // endgroup + // nop // const int instruction_1 = *(int*)(instruction_addr); boint = inv_bo_field(instruction_1); @@ -523,10 +537,10 @@ void MacroAssembler::set_dest_of_bc_far_at(address instruction_addr, address des // variant 1: // // bcxx DEST - // endgroup + // nop // masm.bc(boint, biint, dest); - masm.endgroup(); + masm.nop(); } else { // variant 2: // @@ -810,7 +824,22 @@ void MacroAssembler::save_volatile_gprs(Register dst, int offset) { std(R9, offset, dst); offset += 8; std(R10, offset, dst); offset += 8; std(R11, offset, dst); offset += 8; - std(R12, offset, dst); + std(R12, offset, dst); offset += 8; + + stfd(F0, offset, dst); offset += 8; + stfd(F1, offset, dst); offset += 8; + stfd(F2, offset, dst); offset += 8; + stfd(F3, offset, dst); offset += 8; + stfd(F4, offset, dst); offset += 8; + stfd(F5, offset, dst); offset += 8; + stfd(F6, offset, dst); offset += 8; + stfd(F7, offset, dst); offset += 8; + stfd(F8, offset, dst); offset += 8; + stfd(F9, offset, dst); offset += 8; + stfd(F10, offset, dst); offset += 8; + stfd(F11, offset, dst); offset += 8; + stfd(F12, offset, dst); offset += 8; + stfd(F13, offset, dst); } // For verify_oops. @@ -825,7 +854,22 @@ void MacroAssembler::restore_volatile_gprs(Register src, int offset) { ld(R9, offset, src); offset += 8; ld(R10, offset, src); offset += 8; ld(R11, offset, src); offset += 8; - ld(R12, offset, src); + ld(R12, offset, src); offset += 8; + + lfd(F0, offset, src); offset += 8; + lfd(F1, offset, src); offset += 8; + lfd(F2, offset, src); offset += 8; + lfd(F3, offset, src); offset += 8; + lfd(F4, offset, src); offset += 8; + lfd(F5, offset, src); offset += 8; + lfd(F6, offset, src); offset += 8; + lfd(F7, offset, src); offset += 8; + lfd(F8, offset, src); offset += 8; + lfd(F9, offset, src); offset += 8; + lfd(F10, offset, src); offset += 8; + lfd(F11, offset, src); offset += 8; + lfd(F12, offset, src); offset += 8; + lfd(F13, offset, src); } void MacroAssembler::save_LR_CR(Register tmp) { @@ -908,7 +952,7 @@ void MacroAssembler::push_frame(unsigned int bytes, Register tmp) { if (is_simm(-offset, 16)) { stdu(R1_SP, -offset, R1_SP); } else { - load_const(tmp, -offset); + load_const_optimized(tmp, -offset); stdux(R1_SP, R1_SP, tmp); } } @@ -1090,20 +1134,21 @@ address MacroAssembler::call_c_using_toc(const FunctionDescriptor* fd, assert(fd->entry() != NULL, "function must be linked"); AddressLiteral fd_entry(fd->entry()); - load_const_from_method_toc(R11, fd_entry, toc); + bool success = load_const_from_method_toc(R11, fd_entry, toc, /*fixed_size*/ true); mtctr(R11); if (fd->env() == NULL) { li(R11, 0); nop(); } else { AddressLiteral fd_env(fd->env()); - load_const_from_method_toc(R11, fd_env, toc); + success = success && load_const_from_method_toc(R11, fd_env, toc, /*fixed_size*/ true); } AddressLiteral fd_toc(fd->toc()); - load_toc_from_toc(R2_TOC, fd_toc, toc); - // R2_TOC is killed. + // Set R2_TOC (load from toc) + success = success && load_const_from_method_toc(R2_TOC, fd_toc, toc, /*fixed_size*/ true); bctrl(); _last_calls_return_pc = pc(); + if (!success) { return NULL; } } else { // It's a friend function, load the entry point and don't care about // toc and env. Use an optimizable call instruction, but ensure the @@ -1367,11 +1412,6 @@ void MacroAssembler::cmpxchgw(ConditionRegister flag, Register dest_current_valu bool preset_result_reg = (int_flag_success != dest_current_value && int_flag_success != compare_value && int_flag_success != exchange_value && int_flag_success != addr_base); - // release/fence semantics - if (semantics & MemBarRel) { - release(); - } - if (use_result_reg && preset_result_reg) { li(int_flag_success, 0); // preset (assume cas failed) } @@ -1383,6 +1423,11 @@ void MacroAssembler::cmpxchgw(ConditionRegister flag, Register dest_current_valu bne(flag, failed); } + // release/fence semantics + if (semantics & MemBarRel) { + release(); + } + // atomic emulation loop bind(retry); @@ -1462,11 +1507,6 @@ void MacroAssembler::cmpxchgd(ConditionRegister flag, int_flag_success!=exchange_value && int_flag_success!=addr_base); assert(int_flag_success == noreg || failed_ext == NULL, "cannot have both"); - // release/fence semantics - if (semantics & MemBarRel) { - release(); - } - if (use_result_reg && preset_result_reg) { li(int_flag_success, 0); // preset (assume cas failed) } @@ -1478,6 +1518,11 @@ void MacroAssembler::cmpxchgd(ConditionRegister flag, bne(flag, failed); } + // release/fence semantics + if (semantics & MemBarRel) { + release(); + } + // atomic emulation loop bind(retry); @@ -1501,8 +1546,6 @@ void MacroAssembler::cmpxchgd(ConditionRegister flag, li(int_flag_success, 1); } - // POWER6 doesn't need isync in CAS. - // Always emit isync to be on the safe side. if (semantics & MemBarFenceAfter) { fence(); } else if (semantics & MemBarAcq) { @@ -1627,13 +1670,14 @@ void MacroAssembler::lookup_virtual_method(Register recv_klass, } /////////////////////////////////////////// subtype checking //////////////////////////////////////////// - void MacroAssembler::check_klass_subtype_fast_path(Register sub_klass, Register super_klass, Register temp1_reg, Register temp2_reg, - Label& L_success, - Label& L_failure) { + Label* L_success, + Label* L_failure, + Label* L_slow_path, + RegisterOrConstant super_check_offset) { const Register check_cache_offset = temp1_reg; const Register cached_super = temp2_reg; @@ -1643,6 +1687,18 @@ void MacroAssembler::check_klass_subtype_fast_path(Register sub_klass, int sco_offset = in_bytes(Klass::super_check_offset_offset()); int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); + bool must_load_sco = (super_check_offset.constant_or_zero() == -1); + bool need_slow_path = (must_load_sco || super_check_offset.constant_or_zero() == sco_offset); + + Label L_fallthrough; + int label_nulls = 0; + if (L_success == NULL) { L_success = &L_fallthrough; label_nulls++; } + if (L_failure == NULL) { L_failure = &L_fallthrough; label_nulls++; } + if (L_slow_path == NULL) { L_slow_path = &L_fallthrough; label_nulls++; } + assert(label_nulls <= 1 || + (L_slow_path == &L_fallthrough && label_nulls <= 2 && !need_slow_path), + "at most one NULL in the batch, usually"); + // If the pointers are equal, we are done (e.g., String[] elements). // This self-check enables sharing of secondary supertype arrays among // non-primary types such as array-of-interface. Otherwise, each such @@ -1651,15 +1707,20 @@ void MacroAssembler::check_klass_subtype_fast_path(Register sub_klass, // type checks are in fact trivially successful in this manner, // so we get a nicely predicted branch right at the start of the check. cmpd(CCR0, sub_klass, super_klass); - beq(CCR0, L_success); + beq(CCR0, *L_success); // Check the supertype display: + if (must_load_sco) { + // The super check offset is always positive... lwz(check_cache_offset, sco_offset, super_klass); + super_check_offset = RegisterOrConstant(check_cache_offset); + // super_check_offset is register. + assert_different_registers(sub_klass, super_klass, cached_super, super_check_offset.as_register()); + } // The loaded value is the offset from KlassOopDesc. - ldx(cached_super, check_cache_offset, sub_klass); + ld(cached_super, super_check_offset, sub_klass); cmpd(CCR0, cached_super, super_klass); - beq(CCR0, L_success); // This check has worked decisively for primary supers. // Secondary supers are sought in the super_cache ('super_cache_addr'). @@ -1672,9 +1733,39 @@ void MacroAssembler::check_klass_subtype_fast_path(Register sub_klass, // So if it was a primary super, we can just fail immediately. // Otherwise, it's the slow path for us (no success at this point). - cmpwi(CCR0, check_cache_offset, sc_offset); - bne(CCR0, L_failure); - // bind(slow_path); // fallthru +#define FINAL_JUMP(label) if (&(label) != &L_fallthrough) { b(label); } + + if (super_check_offset.is_register()) { + beq(CCR0, *L_success); + cmpwi(CCR0, super_check_offset.as_register(), sc_offset); + if (L_failure == &L_fallthrough) { + beq(CCR0, *L_slow_path); + } else { + bne(CCR0, *L_failure); + FINAL_JUMP(*L_slow_path); + } + } else { + if (super_check_offset.as_constant() == sc_offset) { + // Need a slow path; fast failure is impossible. + if (L_slow_path == &L_fallthrough) { + beq(CCR0, *L_success); + } else { + bne(CCR0, *L_slow_path); + FINAL_JUMP(*L_success); + } + } else { + // No slow path; it's a fast decision. + if (L_failure == &L_fallthrough) { + beq(CCR0, *L_success); + } else { + bne(CCR0, *L_failure); + FINAL_JUMP(*L_success); + } + } + } + + bind(L_fallthrough); +#undef FINAL_JUMP } void MacroAssembler::check_klass_subtype_slow_path(Register sub_klass, @@ -1698,7 +1789,7 @@ void MacroAssembler::check_klass_subtype_slow_path(Register sub_klass, ld(array_ptr, source_offset, sub_klass); - //assert(4 == arrayOopDesc::length_length_in_bytes(), "precondition violated."); + // TODO: PPC port: assert(4 == arrayOopDesc::length_length_in_bytes(), "precondition violated."); lwz(temp, length_offset, array_ptr); cmpwi(CCR0, temp, 0); beq(CCR0, result_reg!=noreg ? failure : fallthru); // length 0 @@ -1719,8 +1810,9 @@ void MacroAssembler::check_klass_subtype_slow_path(Register sub_klass, bind(hit); std(super_klass, target_offset, sub_klass); // save result to cache - if (result_reg != noreg) li(result_reg, 0); // load zero result (indicates a hit) - if (L_success != NULL) b(*L_success); + if (result_reg != noreg) { li(result_reg, 0); } // load zero result (indicates a hit) + if (L_success != NULL) { b(*L_success); } + else if (result_reg == noreg) { blr(); } // return with CR0.eq if neither label nor result reg provided bind(fallthru); } @@ -1732,7 +1824,7 @@ void MacroAssembler::check_klass_subtype(Register sub_klass, Register temp2_reg, Label& L_success) { Label L_failure; - check_klass_subtype_fast_path(sub_klass, super_klass, temp1_reg, temp2_reg, L_success, L_failure); + check_klass_subtype_fast_path(sub_klass, super_klass, temp1_reg, temp2_reg, &L_success, &L_failure); check_klass_subtype_slow_path(sub_klass, super_klass, temp1_reg, temp2_reg, &L_success); bind(L_failure); // Fallthru if not successful. } @@ -1765,6 +1857,7 @@ RegisterOrConstant MacroAssembler::argument_offset(RegisterOrConstant arg_slot, } } +// Supports temp2_reg = R0. void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj_reg, Register mark_reg, Register temp_reg, Register temp2_reg, Label& done, Label* slow_case) { @@ -1788,10 +1881,10 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj "biased locking makes assumptions about bit layout"); if (PrintBiasedLockingStatistics) { - load_const(temp_reg, (address) BiasedLocking::total_entry_count_addr(), temp2_reg); - lwz(temp2_reg, 0, temp_reg); - addi(temp2_reg, temp2_reg, 1); - stw(temp2_reg, 0, temp_reg); + load_const(temp2_reg, (address) BiasedLocking::total_entry_count_addr(), temp_reg); + lwzx(temp_reg, temp2_reg); + addi(temp_reg, temp_reg, 1); + stwx(temp_reg, temp2_reg); } andi(temp_reg, mark_reg, markOopDesc::biased_lock_mask_in_place); @@ -1809,10 +1902,10 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj if (PrintBiasedLockingStatistics) { Label l; bne(cr_reg, l); - load_const(mark_reg, (address) BiasedLocking::biased_lock_entry_count_addr()); - lwz(temp2_reg, 0, mark_reg); - addi(temp2_reg, temp2_reg, 1); - stw(temp2_reg, 0, mark_reg); + load_const(temp2_reg, (address) BiasedLocking::biased_lock_entry_count_addr()); + lwzx(mark_reg, temp2_reg); + addi(mark_reg, mark_reg, 1); + stwx(mark_reg, temp2_reg); // restore mark_reg ld(mark_reg, oopDesc::mark_offset_in_bytes(), obj_reg); bind(l); @@ -1878,10 +1971,10 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj // need to revoke that bias. The revocation will occur in the // interpreter runtime in the slow case. if (PrintBiasedLockingStatistics) { - load_const(temp_reg, (address) BiasedLocking::anonymously_biased_lock_entry_count_addr(), temp2_reg); - lwz(temp2_reg, 0, temp_reg); - addi(temp2_reg, temp2_reg, 1); - stw(temp2_reg, 0, temp_reg); + load_const(temp2_reg, (address) BiasedLocking::anonymously_biased_lock_entry_count_addr(), temp_reg); + lwzx(temp_reg, temp2_reg); + addi(temp_reg, temp_reg, 1); + stwx(temp_reg, temp2_reg); } b(done); @@ -1892,15 +1985,14 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj // value as the comparison value when doing the cas to acquire the // bias in the current epoch. In other words, we allow transfer of // the bias from one thread to another directly in this situation. - andi(temp_reg, mark_reg, markOopDesc::age_mask_in_place); - orr(temp_reg, R16_thread, temp_reg); - load_klass(temp2_reg, obj_reg); - ld(temp2_reg, in_bytes(Klass::prototype_header_offset()), temp2_reg); - orr(temp_reg, temp_reg, temp2_reg); + load_klass(temp_reg, obj_reg); + andi(temp2_reg, mark_reg, markOopDesc::age_mask_in_place); + orr(temp2_reg, R16_thread, temp2_reg); + ld(temp_reg, in_bytes(Klass::prototype_header_offset()), temp_reg); + orr(temp_reg, temp2_reg, temp_reg); assert(oopDesc::mark_offset_in_bytes() == 0, "offset of _mark is not 0"); - // CmpxchgX sets cr_reg to cmpX(temp2_reg, mark_reg). cmpxchgd(/*flag=*/cr_reg, /*current_value=*/temp2_reg, /*compare_value=*/mark_reg, /*exchange_value=*/temp_reg, /*where=*/obj_reg, @@ -1913,10 +2005,10 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj // need to revoke that bias. The revocation will occur in the // interpreter runtime in the slow case. if (PrintBiasedLockingStatistics) { - load_const(temp_reg, (address) BiasedLocking::rebiased_lock_entry_count_addr(), temp2_reg); - lwz(temp2_reg, 0, temp_reg); - addi(temp2_reg, temp2_reg, 1); - stw(temp2_reg, 0, temp_reg); + load_const(temp2_reg, (address) BiasedLocking::rebiased_lock_entry_count_addr(), temp_reg); + lwzx(temp_reg, temp2_reg); + addi(temp_reg, temp_reg, 1); + stwx(temp_reg, temp2_reg); } b(done); @@ -1952,10 +2044,10 @@ void MacroAssembler::biased_locking_enter(ConditionRegister cr_reg, Register obj if (PrintBiasedLockingStatistics) { Label l; bne(cr_reg, l); - load_const(temp_reg, (address) BiasedLocking::revoked_lock_entry_count_addr(), temp2_reg); - lwz(temp2_reg, 0, temp_reg); - addi(temp2_reg, temp2_reg, 1); - stw(temp2_reg, 0, temp_reg); + load_const(temp2_reg, (address) BiasedLocking::revoked_lock_entry_count_addr(), temp_reg); + lwzx(temp_reg, temp2_reg); + addi(temp_reg, temp_reg, 1); + stwx(temp_reg, temp2_reg); bind(l); } @@ -1977,6 +2069,109 @@ void MacroAssembler::biased_locking_exit (ConditionRegister cr_reg, Register mar beq(cr_reg, done); } +// allocation (for C1) +void MacroAssembler::eden_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Register t2, // temp register + Label& slow_case // continuation point if fast allocation fails +) { + b(slow_case); +} + +void MacroAssembler::tlab_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Label& slow_case // continuation point if fast allocation fails +) { + // make sure arguments make sense + assert_different_registers(obj, var_size_in_bytes, t1); + assert(0 <= con_size_in_bytes && is_simm13(con_size_in_bytes), "illegal object size"); + assert((con_size_in_bytes & MinObjAlignmentInBytesMask) == 0, "object size is not multiple of alignment"); + + const Register new_top = t1; + //verify_tlab(); not implemented + + ld(obj, in_bytes(JavaThread::tlab_top_offset()), R16_thread); + ld(R0, in_bytes(JavaThread::tlab_end_offset()), R16_thread); + if (var_size_in_bytes == noreg) { + addi(new_top, obj, con_size_in_bytes); + } else { + add(new_top, obj, var_size_in_bytes); + } + cmpld(CCR0, new_top, R0); + bc_far_optimized(Assembler::bcondCRbiIs1, bi0(CCR0, Assembler::greater), slow_case); + +#ifdef ASSERT + // make sure new free pointer is properly aligned + { + Label L; + andi_(R0, new_top, MinObjAlignmentInBytesMask); + beq(CCR0, L); + stop("updated TLAB free is not properly aligned", 0x934); + bind(L); + } +#endif // ASSERT + + // update the tlab top pointer + std(new_top, in_bytes(JavaThread::tlab_top_offset()), R16_thread); + //verify_tlab(); not implemented +} +void MacroAssembler::tlab_refill(Label& retry_tlab, Label& try_eden, Label& slow_case) { + unimplemented("tlab_refill"); +} +void MacroAssembler::incr_allocated_bytes(RegisterOrConstant size_in_bytes, Register t1, Register t2) { + unimplemented("incr_allocated_bytes"); +} + +address MacroAssembler::emit_trampoline_stub(int destination_toc_offset, + int insts_call_instruction_offset, Register Rtoc) { + // Start the stub. + address stub = start_a_stub(64); + if (stub == NULL) { return NULL; } // CodeCache full: bail out + + // Create a trampoline stub relocation which relates this trampoline stub + // with the call instruction at insts_call_instruction_offset in the + // instructions code-section. + relocate(trampoline_stub_Relocation::spec(code()->insts()->start() + insts_call_instruction_offset)); + const int stub_start_offset = offset(); + + // For java_to_interp stubs we use R11_scratch1 as scratch register + // and in call trampoline stubs we use R12_scratch2. This way we + // can distinguish them (see is_NativeCallTrampolineStub_at()). + Register reg_scratch = R12_scratch2; + + // Now, create the trampoline stub's code: + // - load the TOC + // - load the call target from the constant pool + // - call + if (Rtoc == noreg) { + calculate_address_from_global_toc(reg_scratch, method_toc()); + Rtoc = reg_scratch; + } + + ld_largeoffset_unchecked(reg_scratch, destination_toc_offset, Rtoc, false); + mtctr(reg_scratch); + bctr(); + + const address stub_start_addr = addr_at(stub_start_offset); + + // Assert that the encoded destination_toc_offset can be identified and that it is correct. + assert(destination_toc_offset == NativeCallTrampolineStub_at(stub_start_addr)->destination_toc_offset(), + "encoded offset into the constant pool must match"); + // Trampoline_stub_size should be good. + assert((uint)(offset() - stub_start_offset) <= trampoline_stub_size, "should be good size"); + assert(is_NativeCallTrampolineStub_at(stub_start_addr), "doesn't look like a trampoline"); + + // End the stub. + end_a_stub(); + return stub; +} + // TM on PPC64. void MacroAssembler::atomic_inc_ptr(Register addr, Register result, int simm16) { Label retry; @@ -2387,17 +2582,16 @@ void MacroAssembler::compiler_fast_lock_object(ConditionRegister flag, Register // Must fence, otherwise, preceding store(s) may float below cmpxchg. // Compare object markOop with mark and if equal exchange scratch1 with object markOop. - // CmpxchgX sets cr_reg to cmpX(current, displaced). - membar(Assembler::StoreStore); cmpxchgd(/*flag=*/flag, /*current_value=*/current_header, /*compare_value=*/displaced_header, /*exchange_value=*/box, /*where=*/oop, - MacroAssembler::MemBarAcq, + MacroAssembler::MemBarRel | MacroAssembler::MemBarAcq, MacroAssembler::cmpxchgx_hint_acquire_lock(), noreg, - &cas_failed); + &cas_failed, + /*check without membar and ldarx first*/true); assert(oopDesc::mark_offset_in_bytes() == 0, "offset of _mark is not 0"); // If the compare-and-exchange succeeded, then we found an unlocked @@ -2410,8 +2604,7 @@ void MacroAssembler::compiler_fast_lock_object(ConditionRegister flag, Register // Check if the owner is self by comparing the value in the markOop of object // (current_header) with the stack pointer. sub(current_header, current_header, R1_SP); - load_const_optimized(temp, (address) (~(os::vm_page_size()-1) | - markOopDesc::lock_mask_in_place)); + load_const_optimized(temp, ~(os::vm_page_size()-1) | markOopDesc::lock_mask_in_place); and_(R0/*==0?*/, current_header, temp); // If condition is true we are cont and hence we can store 0 as the @@ -2437,8 +2630,6 @@ void MacroAssembler::compiler_fast_lock_object(ConditionRegister flag, Register // Try to CAS m->owner from NULL to current thread. addi(temp, displaced_header, ObjectMonitor::owner_offset_in_bytes()-markOopDesc::monitor_value); - li(displaced_header, 0); - // CmpxchgX sets flag to cmpX(current, displaced). cmpxchgd(/*flag=*/flag, /*current_value=*/current_header, /*compare_value=*/(intptr_t)0, @@ -2924,31 +3115,12 @@ void MacroAssembler::load_klass(Register dst, Register src) { } } -void MacroAssembler::load_klass_with_trap_null_check(Register dst, Register src) { - if (!os::zero_page_read_protected()) { - if (TrapBasedNullChecks) { - trap_null_check(src); - } - } - load_klass(dst, src); -} - -void MacroAssembler::reinit_heapbase(Register d, Register tmp) { - if (Universe::heap() != NULL) { - load_const_optimized(R30, Universe::narrow_ptrs_base(), tmp); - } else { - // Heap not yet allocated. Load indirectly. - int simm16_offset = load_const_optimized(R30, Universe::narrow_ptrs_base_addr(), tmp, true); - ld(R30, simm16_offset, R30); - } -} - // Clear Array // Kills both input registers. tmp == R0 is allowed. void MacroAssembler::clear_memory_doubleword(Register base_ptr, Register cnt_dwords, Register tmp) { // Procedure for large arrays (uses data cache block zero instruction). Label startloop, fast, fastloop, small_rest, restloop, done; - const int cl_size = VM_Version::get_cache_line_size(), + const int cl_size = VM_Version::L1_data_cache_line_size(), cl_dwords = cl_size>>3, cl_dw_addr_bits = exact_log2(cl_dwords), dcbz_min = 1; // Min count of dcbz executions, needs to be >0. @@ -4021,7 +4193,7 @@ void MacroAssembler::multiply_128_x_128_loop(Register x_xstart, bind(L_check_1); addi(idx, idx, 0x2); - andi_(idx, idx, 0x1) ; + andi_(idx, idx, 0x1); addic_(idx, idx, -1); blt(CCR0, L_post_third_loop_done); @@ -4251,17 +4423,42 @@ void MacroAssembler::verify_oop(Register oop, const char* msg) { address/* FunctionDescriptor** */fd = StubRoutines::verify_oop_subroutine_entry_address(); const Register tmp = R11; // Will be preserved. - const int nbytes_save = 11*8; // Volatile gprs except R0. + const int nbytes_save = MacroAssembler::num_volatile_regs * 8; save_volatile_gprs(R1_SP, -nbytes_save); // except R0 - if (oop == tmp) mr(R4_ARG2, oop); + mr_if_needed(R4_ARG2, oop); + save_LR_CR(tmp); // save in old frame + push_frame_reg_args(nbytes_save, tmp); + // load FunctionDescriptor** / entry_address * + load_const_optimized(tmp, fd, R0); + // load FunctionDescriptor* / entry_address + ld(tmp, 0, tmp); + load_const_optimized(R3_ARG1, (address)msg, R0); + // Call destination for its side effect. + call_c(tmp); + + pop_frame(); + restore_LR_CR(tmp); + restore_volatile_gprs(R1_SP, -nbytes_save); // except R0 +} + +void MacroAssembler::verify_oop_addr(RegisterOrConstant offs, Register base, const char* msg) { + if (!VerifyOops) { + return; + } + + address/* FunctionDescriptor** */fd = StubRoutines::verify_oop_subroutine_entry_address(); + const Register tmp = R11; // Will be preserved. + const int nbytes_save = MacroAssembler::num_volatile_regs * 8; + save_volatile_gprs(R1_SP, -nbytes_save); // except R0 + + ld(R4_ARG2, offs, base); save_LR_CR(tmp); // save in old frame push_frame_reg_args(nbytes_save, tmp); // load FunctionDescriptor** / entry_address * load_const_optimized(tmp, fd, R0); // load FunctionDescriptor* / entry_address ld(tmp, 0, tmp); - if (oop != tmp) mr_if_needed(R4_ARG2, oop); load_const_optimized(R3_ARG1, (address)msg, R0); // Call destination for its side effect. call_c(tmp); diff --git a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.hpp b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.hpp index 37930a4bfc2..df58832b160 100644 --- a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.hpp @@ -119,11 +119,8 @@ class MacroAssembler: public Assembler { // Emits an oop const to the constant pool, loads the constant, and // sets a relocation info with address current_pc. - void load_const_from_method_toc(Register dst, AddressLiteral& a, Register toc); - void load_toc_from_toc(Register dst, AddressLiteral& a, Register toc) { - assert(dst == R2_TOC, "base register must be TOC"); - load_const_from_method_toc(dst, a, toc); - } + // Returns true if successful. + bool load_const_from_method_toc(Register dst, AddressLiteral& a, Register toc, bool fixed_size = false); static bool is_load_const_from_method_toc_at(address a); static int get_offset_of_load_const_from_method_toc_at(address a); @@ -174,6 +171,7 @@ class MacroAssembler: public Assembler { // optimize: flag for telling the conditional far branch to optimize // itself when relocated. void bc_far(int boint, int biint, Label& dest, int optimize); + void bc_far_optimized(int boint, int biint, Label& dest); // 1 or 2 instructions // Relocation of conditional far branches. static bool is_bc_far_at(address instruction_addr); static address get_dest_of_bc_far_at(address instruction_addr); @@ -262,6 +260,7 @@ class MacroAssembler: public Assembler { // some ABI-related functions void save_nonvolatile_gprs( Register dst_base, int offset); void restore_nonvolatile_gprs(Register src_base, int offset); + enum { num_volatile_regs = 11 + 14 }; // GPR + FPR void save_volatile_gprs( Register dst_base, int offset); void restore_volatile_gprs(Register src_base, int offset); void save_LR_CR( Register tmp); // tmp contains LR on return. @@ -461,8 +460,10 @@ class MacroAssembler: public Assembler { Register super_klass, Register temp1_reg, Register temp2_reg, - Label& L_success, - Label& L_failure); + Label* L_success, + Label* L_failure, + Label* L_slow_path = NULL, // default fall through + RegisterOrConstant super_check_offset = RegisterOrConstant(-1)); // The rest of the type check; must be wired to a corresponding fast path. // It does not repeat the fast path logic, so don't use it standalone. @@ -507,6 +508,28 @@ class MacroAssembler: public Assembler { // biased locking exit case failed. void biased_locking_exit(ConditionRegister cr_reg, Register mark_addr, Register temp_reg, Label& done); + // allocation (for C1) + void eden_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Register t2, // temp register + Label& slow_case // continuation point if fast allocation fails + ); + void tlab_allocate( + Register obj, // result: pointer to object after successful allocation + Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise + int con_size_in_bytes, // object size in bytes if known at compile time + Register t1, // temp register + Label& slow_case // continuation point if fast allocation fails + ); + void tlab_refill(Label& retry_tlab, Label& try_eden, Label& slow_case); + void incr_allocated_bytes(RegisterOrConstant size_in_bytes, Register t1, Register t2); + + enum { trampoline_stub_size = 6 * 4 }; + address emit_trampoline_stub(int destination_toc_offset, int insts_call_instruction_offset, Register Rtoc = noreg); + void atomic_inc_ptr(Register addr, Register result, int simm16 = 1); void atomic_ori_int(Register addr, Register result, int uimm16); @@ -597,9 +620,7 @@ class MacroAssembler: public Assembler { // Implicit or explicit null check, jumps to static address exception_entry. inline void null_check_throw(Register a, int offset, Register temp_reg, address exception_entry); - - // Check accessed object for null. Use SIGTRAP-based null checks on AIX. - inline void load_with_trap_null_check(Register d, int si16, Register s1); + inline void null_check(Register a, int offset, Label *Lis_null); // implicit only if Lis_null not provided // Load heap oop and decompress. Loaded oop may not be null. // Specify tmp to save one cycle. @@ -619,20 +640,17 @@ class MacroAssembler: public Assembler { inline Register decode_heap_oop_not_null(Register d, Register src = noreg); // Null allowed. + inline Register encode_heap_oop(Register d, Register src); // Prefer null check in GC barrier! inline void decode_heap_oop(Register d); // Load/Store klass oop from klass field. Compress. void load_klass(Register dst, Register src); - void load_klass_with_trap_null_check(Register dst, Register src); void store_klass(Register dst_oop, Register klass, Register tmp = R0); void store_klass_gap(Register dst_oop, Register val = noreg); // Will store 0 if val not specified. static int instr_size_for_decode_klass_not_null(); void decode_klass_not_null(Register dst, Register src = noreg); Register encode_klass_not_null(Register dst, Register src = noreg); - // Load common heap base into register. - void reinit_heapbase(Register d, Register tmp = noreg); - // SIGTRAP-based range checks for arrays. inline void trap_range_check_l(Register a, Register b); inline void trap_range_check_l(Register a, int si16); @@ -750,6 +768,7 @@ class MacroAssembler: public Assembler { // Emit code to verify that reg contains a valid oop if +VerifyOops is set. void verify_oop(Register reg, const char* s = "broken oop"); + void verify_oop_addr(RegisterOrConstant offs, Register base, const char* s = "contains broken oop"); // TODO: verify method and klass metadata (compare against vptr?) void _verify_method_ptr(Register reg, const char * msg, const char * file, int line) {} diff --git a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.inline.hpp b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.inline.hpp index 9d062d799c7..62843482074 100644 --- a/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.inline.hpp +++ b/hotspot/src/cpu/ppc/vm/macroAssembler_ppc.inline.hpp @@ -70,9 +70,11 @@ inline void MacroAssembler::endgroup_if_needed(bool needed) { } inline void MacroAssembler::membar(int bits) { - // TODO: use elemental_membar(bits) for Power 8 and disable optimization of acquire-release - // (Matcher::post_membar_release where we use PPC64_ONLY(xop == Op_MemBarRelease ||)) - if (bits & StoreLoad) sync(); else lwsync(); + // Comment: Usage of elemental_membar(bits) is not recommended for Power 8. + // If elemental_membar(bits) is used, disable optimization of acquire-release + // (Matcher::post_membar_release where we use PPC64_ONLY(xop == Op_MemBarRelease ||))! + if (bits & StoreLoad) { sync(); } + else if (bits) { lwsync(); } } inline void MacroAssembler::release() { membar(LoadStore | StoreStore); } inline void MacroAssembler::acquire() { membar(LoadLoad | LoadStore); } @@ -86,7 +88,7 @@ inline address MacroAssembler::global_toc() { // Offset of given address to the global TOC. inline int MacroAssembler::offset_to_global_toc(const address addr) { intptr_t offset = (intptr_t)addr - (intptr_t)MacroAssembler::global_toc(); - assert(Assembler::is_simm((long)offset, 31) && offset >= 0, "must be in range"); + assert(Assembler::is_uimm((long)offset, 31), "must be in range"); return (int)offset; } @@ -98,7 +100,7 @@ inline address MacroAssembler::method_toc() { // Offset of given address to current method's TOC. inline int MacroAssembler::offset_to_method_toc(address addr) { intptr_t offset = (intptr_t)addr - (intptr_t)method_toc(); - assert(is_simm((long)offset, 31) && offset >= 0, "must be in range"); + assert(Assembler::is_uimm((long)offset, 31), "must be in range"); return (int)offset; } @@ -190,13 +192,13 @@ inline bool MacroAssembler::is_bc_far_variant1_at(address instruction_addr) { // Variant 1, the 1st instruction contains the destination address: // // bcxx DEST - // endgroup + // nop // const int instruction_1 = *(int*)(instruction_addr); const int instruction_2 = *(int*)(instruction_addr + 4); return is_bcxx(instruction_1) && (inv_bd_field(instruction_1, (intptr_t)instruction_addr) != (intptr_t)(instruction_addr + 2*4)) && - is_endgroup(instruction_2); + is_nop(instruction_2); } // Relocation of conditional far branches. @@ -302,13 +304,17 @@ inline void MacroAssembler::null_check_throw(Register a, int offset, Register te } } -inline void MacroAssembler::load_with_trap_null_check(Register d, int si16, Register s1) { - if (!os::zero_page_read_protected()) { +inline void MacroAssembler::null_check(Register a, int offset, Label *Lis_null) { + if (!ImplicitNullChecks || needs_explicit_null_check(offset) || !os::zero_page_read_protected()) { if (TrapBasedNullChecks) { - trap_null_check(s1); + assert(UseSIGTRAP, "sanity"); + trap_null_check(a); + } else if (Lis_null){ + Label ok; + cmpdi(CCR0, a, 0); + beq(CCR0, *Lis_null); } } - ld(d, si16, s1); } inline void MacroAssembler::load_heap_oop_not_null(Register d, RegisterOrConstant offs, Register s1, Register tmp) { @@ -365,6 +371,26 @@ inline Register MacroAssembler::encode_heap_oop_not_null(Register d, Register sr return current; // Encoded oop is in this register. } +inline Register MacroAssembler::encode_heap_oop(Register d, Register src) { + if (Universe::narrow_oop_base() != NULL) { + if (VM_Version::has_isel()) { + cmpdi(CCR0, src, 0); + Register co = encode_heap_oop_not_null(d, src); + assert(co == d, "sanity"); + isel_0(d, CCR0, Assembler::equal); + } else { + Label isNull; + or_(d, src, src); // move and compare 0 + beq(CCR0, isNull); + encode_heap_oop_not_null(d, src); + bind(isNull); + } + return d; + } else { + return encode_heap_oop_not_null(d, src); + } +} + inline Register MacroAssembler::decode_heap_oop_not_null(Register d, Register src) { if (Universe::narrow_oop_base_disjoint() && src != noreg && src != d && Universe::narrow_oop_shift() != 0) { diff --git a/hotspot/src/cpu/ppc/vm/methodHandles_ppc.cpp b/hotspot/src/cpu/ppc/vm/methodHandles_ppc.cpp index 168aacb6326..12e10dbc573 100644 --- a/hotspot/src/cpu/ppc/vm/methodHandles_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/methodHandles_ppc.cpp @@ -1,6 +1,6 @@ /* * Copyright (c) 1997, 2015, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2015 SAP SE. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -502,8 +502,7 @@ void trace_method_handle_stub(const char* adaptername, frame cur_frame = os::current_frame(); // Robust search of trace_calling_frame (independant of inlining). - // Assumes saved_regs comes from a pusha in the trace_calling_frame. - assert(cur_frame.sp() < saved_regs, "registers not saved on stack ?"); + assert(cur_frame.sp() <= saved_regs, "registers not saved on stack ?"); frame trace_calling_frame = os::get_sender_for_C_frame(&cur_frame); while (trace_calling_frame.fp() < saved_regs) { trace_calling_frame = os::get_sender_for_C_frame(&trace_calling_frame); @@ -537,7 +536,7 @@ void MethodHandles::trace_method_handle(MacroAssembler* _masm, const char* adapt BLOCK_COMMENT("trace_method_handle {"); const Register tmp = R11; // Will be preserved. - const int nbytes_save = 11*8; // volatile gprs except R0 + const int nbytes_save = MacroAssembler::num_volatile_regs * 8; __ save_volatile_gprs(R1_SP, -nbytes_save); // except R0 __ save_LR_CR(tmp); // save in old frame diff --git a/hotspot/src/cpu/ppc/vm/nativeInst_ppc.cpp b/hotspot/src/cpu/ppc/vm/nativeInst_ppc.cpp index ecca49af2ef..37956925d0d 100644 --- a/hotspot/src/cpu/ppc/vm/nativeInst_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/nativeInst_ppc.cpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2014 SAP AG. All rights reserved. + * Copyright (c) 1997, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -65,13 +65,17 @@ address NativeCall::destination() const { address destination = Assembler::bxx_destination(addr); // Do we use a trampoline stub for this call? - CodeBlob* cb = CodeCache::find_blob_unsafe(addr); // Else we get assertion if nmethod is zombie. - assert(cb && cb->is_nmethod(), "sanity"); - nmethod *nm = (nmethod *)cb; - if (nm->stub_contains(destination) && is_NativeCallTrampolineStub_at(destination)) { - // Yes we do, so get the destination from the trampoline stub. - const address trampoline_stub_addr = destination; - destination = NativeCallTrampolineStub_at(trampoline_stub_addr)->destination(nm); + // Trampoline stubs are located behind the main code. + if (destination > addr) { + // Filter out recursive method invocation (call to verified/unverified entry point). + CodeBlob* cb = CodeCache::find_blob_unsafe(addr); // Else we get assertion if nmethod is zombie. + assert(cb && cb->is_nmethod(), "sanity"); + nmethod *nm = (nmethod *)cb; + if (nm->stub_contains(destination) && is_NativeCallTrampolineStub_at(destination)) { + // Yes we do, so get the destination from the trampoline stub. + const address trampoline_stub_addr = destination; + destination = NativeCallTrampolineStub_at(trampoline_stub_addr)->destination(nm); + } } return destination; @@ -267,7 +271,7 @@ void NativeMovConstReg::set_data(intptr_t data) { oop_addr = r->oop_addr(); *oop_addr = cast_to_oop(data); } else { - assert(oop_addr == r->oop_addr(), "must be only one set-oop here") ; + assert(oop_addr == r->oop_addr(), "must be only one set-oop here"); } } if (iter.type() == relocInfo::metadata_type) { @@ -351,6 +355,27 @@ void NativeJump::verify() { } #endif // ASSERT + +void NativeGeneralJump::insert_unconditional(address code_pos, address entry) { + CodeBuffer cb(code_pos, BytesPerInstWord + 1); + MacroAssembler* a = new MacroAssembler(&cb); + a->b(entry); + ICache::ppc64_flush_icache_bytes(code_pos, NativeGeneralJump::instruction_size); +} + +// MT-safe patching of a jmp instruction. +void NativeGeneralJump::replace_mt_safe(address instr_addr, address code_buffer) { + // Bytes beyond offset NativeGeneralJump::instruction_size are copied by caller. + + // Finally patch out the jump. + volatile juint *jump_addr = (volatile juint*)instr_addr; + // Release not needed because caller uses invalidate_range after copying the remaining bytes. + //OrderAccess::release_store(jump_addr, *((juint*)code_buffer)); + *jump_addr = *((juint*)code_buffer); // atomically store code over branch instruction + ICache::ppc64_flush_icache_bytes(instr_addr, NativeGeneralJump::instruction_size); +} + + //------------------------------------------------------------------- // Call trampoline stubs. @@ -364,10 +389,12 @@ void NativeJump::verify() { // address NativeCallTrampolineStub::encoded_destination_addr() const { - address instruction_addr = addr_at(2 * BytesPerInstWord); - assert(MacroAssembler::is_ld_largeoffset(instruction_addr), - "must be a ld with large offset (from the constant pool)"); - + address instruction_addr = addr_at(0 * BytesPerInstWord); + if (!MacroAssembler::is_ld_largeoffset(instruction_addr)) { + instruction_addr = addr_at(2 * BytesPerInstWord); + assert(MacroAssembler::is_ld_largeoffset(instruction_addr), + "must be a ld with large offset (from the constant pool)"); + } return instruction_addr; } diff --git a/hotspot/src/cpu/ppc/vm/nativeInst_ppc.hpp b/hotspot/src/cpu/ppc/vm/nativeInst_ppc.hpp index 0ef57be0556..f0f6b6a87d9 100644 --- a/hotspot/src/cpu/ppc/vm/nativeInst_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/nativeInst_ppc.hpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 2002, 2013, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2013 SAP AG. All rights reserved. + * Copyright (c) 2002, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -50,6 +50,8 @@ class NativeInstruction VALUE_OBJ_CLASS_SPEC { friend class Relocation; public: + bool is_jump() { return Assembler::is_b(long_at(0)); } // See NativeGeneralJump. + bool is_sigtrap_ic_miss_check() { assert(UseSIGTRAP, "precondition"); return MacroAssembler::is_trap_ic_miss_check(long_at(0)); @@ -235,8 +237,8 @@ inline NativeFarCall* nativeFarCall_at(address instr) { return call; } -// An interface for accessing/manipulating native set_oop imm, reg instructions. -// (used to manipulate inlined data references, etc.) +// An interface for accessing/manipulating native set_oop imm, reg instructions +// (used to manipulate inlined data references, etc.). class NativeMovConstReg: public NativeInstruction { public: @@ -384,10 +386,21 @@ class NativeCallTrampolineStub : public NativeInstruction { void set_destination(address new_destination); }; +// Note: Other stubs must not begin with this pattern. inline bool is_NativeCallTrampolineStub_at(address address) { int first_instr = *(int*)address; - return Assembler::is_addis(first_instr) && - (Register)(intptr_t)Assembler::inv_rt_field(first_instr) == R12_scratch2; + // calculate_address_from_global_toc and long form of ld_largeoffset_unchecked begin with addis with target R12 + if (Assembler::is_addis(first_instr) && + (Register)(intptr_t)Assembler::inv_rt_field(first_instr) == R12_scratch2) return true; + + // short form of ld_largeoffset_unchecked is ld which is followed by mtctr + int second_instr = *((int*)address + 1); + if (Assembler::is_ld(first_instr) && + (Register)(intptr_t)Assembler::inv_rt_field(first_instr) == R12_scratch2 && + Assembler::is_mtctr(second_instr) && + (Register)(intptr_t)Assembler::inv_rs_field(second_instr) == R12_scratch2) return true; + + return false; } inline NativeCallTrampolineStub* NativeCallTrampolineStub_at(address address) { @@ -395,4 +408,102 @@ inline NativeCallTrampolineStub* NativeCallTrampolineStub_at(address address) { return (NativeCallTrampolineStub*)address; } +/////////////////////////////////////////////////////////////////////////////////////////////////// + +//------------------------------------- +// N a t i v e G e n e r a l J u m p +//------------------------------------- + +// Despite the name, handles only simple branches. +class NativeGeneralJump; +inline NativeGeneralJump* nativeGeneralJump_at(address address); + +// Currently only implemented as single unconditional branch. +class NativeGeneralJump: public NativeInstruction { + public: + + enum PPC64_specific_constants { + instruction_size = 4 + }; + + address instruction_address() const { return addr_at(0); } + + // Creation. + friend inline NativeGeneralJump* nativeGeneralJump_at(address addr) { + NativeGeneralJump* jump = (NativeGeneralJump*)(addr); + DEBUG_ONLY( jump->verify(); ) + return jump; + } + + // Insertion of native general jump instruction. + static void insert_unconditional(address code_pos, address entry); + + address jump_destination() const { + DEBUG_ONLY( verify(); ) + return addr_at(0) + Assembler::inv_li_field(long_at(0)); + } + + void set_jump_destination(address dest) { + DEBUG_ONLY( verify(); ) + insert_unconditional(addr_at(0), dest); + } + + static void replace_mt_safe(address instr_addr, address code_buffer); + + void verify() const { guarantee(Assembler::is_b(long_at(0)), "invalid NativeGeneralJump"); } +}; + +// An interface for accessing/manipulating native load int (load_const32). +class NativeMovRegMem; +inline NativeMovRegMem* nativeMovRegMem_at(address address); +class NativeMovRegMem: public NativeInstruction { + public: + + enum PPC64_specific_constants { + instruction_size = 8 + }; + + address instruction_address() const { return addr_at(0); } + + intptr_t offset() const { +#ifdef VM_LITTLE_ENDIAN + short *hi_ptr = (short*)(addr_at(0)); + short *lo_ptr = (short*)(addr_at(4)); +#else + short *hi_ptr = (short*)(addr_at(0) + 2); + short *lo_ptr = (short*)(addr_at(4) + 2); +#endif + return ((*hi_ptr) << 16) | ((*lo_ptr) & 0xFFFF); + } + + void set_offset(intptr_t x) { +#ifdef VM_LITTLE_ENDIAN + short *hi_ptr = (short*)(addr_at(0)); + short *lo_ptr = (short*)(addr_at(4)); +#else + short *hi_ptr = (short*)(addr_at(0) + 2); + short *lo_ptr = (short*)(addr_at(4) + 2); +#endif + *hi_ptr = x >> 16; + *lo_ptr = x & 0xFFFF; + ICache::ppc64_flush_icache_bytes(addr_at(0), NativeMovRegMem::instruction_size); + } + + void add_offset_in_bytes(intptr_t radd_offset) { + set_offset(offset() + radd_offset); + } + + void verify() const { + guarantee(Assembler::is_lis(long_at(0)), "load_const32 1st instr"); + guarantee(Assembler::is_ori(long_at(4)), "load_const32 2nd instr"); + } + + private: + friend inline NativeMovRegMem* nativeMovRegMem_at(address address) { + NativeMovRegMem* test = (NativeMovRegMem*)address; + DEBUG_ONLY( test->verify(); ) + return test; + } +}; + #endif // CPU_PPC_VM_NATIVEINST_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/ppc.ad b/hotspot/src/cpu/ppc/vm/ppc.ad index 10598812c5c..001b539399f 100644 --- a/hotspot/src/cpu/ppc/vm/ppc.ad +++ b/hotspot/src/cpu/ppc/vm/ppc.ad @@ -698,7 +698,7 @@ reg_class ctr_reg(SR_CTR); // ---------------------------- reg_class flt_reg( -/*F0*/ // scratch + F0, F1, F2, F3, @@ -735,7 +735,7 @@ reg_class flt_reg( // Double precision float registers have virtual `high halves' that // are needed by the allocator. reg_class dbl_reg( -/*F0, F0_H*/ // scratch + F0, F0_H, F1, F1_H, F2, F2_H, F3, F3_H, @@ -1040,8 +1040,6 @@ source_hpp %{ // Header information of the source block. //---< Used for optimization in Compile::Shorten_branches >--- //-------------------------------------------------------------- -const uint trampoline_stub_size = 6 * BytesPerInstWord; - class CallStubImpl { public: @@ -1053,7 +1051,7 @@ class CallStubImpl { // This doesn't need to be accurate to the byte, but it // must be larger than or equal to the real size of the stub. static uint size_call_trampoline() { - return trampoline_stub_size; + return MacroAssembler::trampoline_stub_size; } // number of relocations needed by a call trampoline stub @@ -1079,46 +1077,10 @@ source %{ // branch via CTR (LR/link still points to the call-site above) void CallStubImpl::emit_trampoline_stub(MacroAssembler &_masm, int destination_toc_offset, int insts_call_instruction_offset) { - // Start the stub. - address stub = __ start_a_stub(Compile::MAX_stubs_size/2); + address stub = __ emit_trampoline_stub(destination_toc_offset, insts_call_instruction_offset); if (stub == NULL) { - ciEnv::current()->record_failure("CodeCache is full"); - return; + ciEnv::current()->record_out_of_memory_failure(); } - - // For java_to_interp stubs we use R11_scratch1 as scratch register - // and in call trampoline stubs we use R12_scratch2. This way we - // can distinguish them (see is_NativeCallTrampolineStub_at()). - Register reg_scratch = R12_scratch2; - - // Create a trampoline stub relocation which relates this trampoline stub - // with the call instruction at insts_call_instruction_offset in the - // instructions code-section. - __ relocate(trampoline_stub_Relocation::spec(__ code()->insts()->start() + insts_call_instruction_offset)); - const int stub_start_offset = __ offset(); - - // Now, create the trampoline stub's code: - // - load the TOC - // - load the call target from the constant pool - // - call - __ calculate_address_from_global_toc(reg_scratch, __ method_toc()); - __ ld_largeoffset_unchecked(reg_scratch, destination_toc_offset, reg_scratch, false); - __ mtctr(reg_scratch); - __ bctr(); - - const address stub_start_addr = __ addr_at(stub_start_offset); - - // FIXME: Assert that the trampoline stub can be identified and patched. - - // Assert that the encoded destination_toc_offset can be identified and that it is correct. - assert(destination_toc_offset == NativeCallTrampolineStub_at(stub_start_addr)->destination_toc_offset(), - "encoded offset into the constant pool must match"); - // Trampoline_stub_size should be good. - assert((uint)(__ offset() - stub_start_offset) <= trampoline_stub_size, "should be good size"); - assert(is_NativeCallTrampolineStub_at(stub_start_addr), "doesn't look like a trampoline"); - - // End the stub. - __ end_a_stub(); } //============================================================================= @@ -1156,6 +1118,10 @@ EmitCallOffsets emit_call_with_trampoline_stub(MacroAssembler &_masm, address en if (!Compile::current()->in_scratch_emit_size()) { // Put the entry point as a constant into the constant pool. const address entry_point_toc_addr = __ address_constant(entry_point, RelocationHolder::none); + if (entry_point_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return offsets; + } const int entry_point_toc_offset = __ offset_to_method_toc(entry_point_toc_addr); // Emit the trampoline stub which will be related to the branch-and-link below. @@ -2474,6 +2440,10 @@ encode %{ // Create a non-oop constant, no relocation needed. // If it is an IC, it has a virtual_call_Relocation. const_toc_addr = __ long_constant((jlong)$src$$constant); + if (const_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } // Get the constant's TOC offset. toc_offset = __ offset_to_method_toc(const_toc_addr); @@ -2495,6 +2465,10 @@ encode %{ // Create a non-oop constant, no relocation needed. // If it is an IC, it has a virtual_call_Relocation. const_toc_addr = __ long_constant((jlong)$src$$constant); + if (const_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } // Get the constant's TOC offset. const int toc_offset = __ offset_to_method_toc(const_toc_addr); @@ -2631,6 +2605,10 @@ encode %{ const_toc_addr = __ long_constant((jlong)$src$$constant); } + if (const_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } // Get the constant's TOC offset. toc_offset = __ offset_to_method_toc(const_toc_addr); } @@ -2660,6 +2638,10 @@ encode %{ const_toc_addr = __ long_constant((jlong)$src$$constant); } + if (const_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } // Get the constant's TOC offset. const int toc_offset = __ offset_to_method_toc(const_toc_addr); // Store the toc offset of the constant. @@ -3408,13 +3390,19 @@ encode %{ // Put the entry point as a constant into the constant pool. const address entry_point_toc_addr = __ address_constant(entry_point, RelocationHolder::none); + if (entry_point_toc_addr == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } const int entry_point_toc_offset = __ offset_to_method_toc(entry_point_toc_addr); + // Emit the trampoline stub which will be related to the branch-and-link below. CallStubImpl::emit_trampoline_stub(_masm, entry_point_toc_offset, start_offset); if (ciEnv::current()->failing()) { return; } // Code cache may be full. - __ relocate(_optimized_virtual ? - relocInfo::opt_virtual_call_type : relocInfo::static_call_type); + int method_index = resolved_method_index(cbuf); + __ relocate(_optimized_virtual ? opt_virtual_call_Relocation::spec(method_index) + : static_call_Relocation::spec(method_index)); } // The real call. @@ -3433,76 +3421,6 @@ encode %{ } %} - // Emit a method handle call. - // - // Method handle calls from compiled to compiled are going thru a - // c2i -> i2c adapter, extending the frame for their arguments. The - // caller however, returns directly to the compiled callee, that has - // to cope with the extended frame. We restore the original frame by - // loading the callers sp and adding the calculated framesize. - enc_class enc_java_handle_call(method meth) %{ - // TODO: PPC port $archOpcode(ppc64Opcode_compound); - - MacroAssembler _masm(&cbuf); - address entry_point = (address)$meth$$method; - - // Remember the offset not the address. - const int start_offset = __ offset(); - // The trampoline stub. - if (!ra_->C->in_scratch_emit_size()) { - // No entry point given, use the current pc. - // Make sure branch fits into - if (entry_point == 0) entry_point = __ pc(); - - // Put the entry point as a constant into the constant pool. - const address entry_point_toc_addr = __ address_constant(entry_point, RelocationHolder::none); - const int entry_point_toc_offset = __ offset_to_method_toc(entry_point_toc_addr); - - // Emit the trampoline stub which will be related to the branch-and-link below. - CallStubImpl::emit_trampoline_stub(_masm, entry_point_toc_offset, start_offset); - if (ra_->C->env()->failing()) { return; } // Code cache may be full. - assert(_optimized_virtual, "methodHandle call should be a virtual call"); - __ relocate(relocInfo::opt_virtual_call_type); - } - - // The real call. - // Note: At this point we do not have the address of the trampoline - // stub, and the entry point might be too far away for bl, so __ pc() - // serves as dummy and the bl will be patched later. - cbuf.set_insts_mark(); - __ bl(__ pc()); // Emits a relocation. - - assert(_method, "execute next statement conditionally"); - // The stub for call to interpreter. - address stub = CompiledStaticCall::emit_to_interp_stub(cbuf); - if (stub == NULL) { - ciEnv::current()->record_failure("CodeCache is full"); - return; - } - - // Restore original sp. - __ ld(R11_scratch1, 0, R1_SP); // Load caller sp. - const long framesize = ra_->C->frame_slots() << LogBytesPerInt; - unsigned int bytes = (unsigned int)framesize; - long offset = Assembler::align_addr(bytes, frame::alignment_in_bytes); - if (Assembler::is_simm(-offset, 16)) { - __ addi(R1_SP, R11_scratch1, -offset); - } else { - __ load_const_optimized(R12_scratch2, -offset); - __ add(R1_SP, R11_scratch1, R12_scratch2); - } -#ifdef ASSERT - __ ld(R12_scratch2, 0, R1_SP); // Load from unextended_sp. - __ cmpd(CCR0, R11_scratch1, R12_scratch2); - __ asm_assert_eq("backlink changed", 0x8000); -#endif - // If fails should store backlink before unextending. - - if (ra_->C->env()->failing()) { - return; - } - %} - // Second node of expanded dynamic call - the call. enc_class enc_java_dynamic_call_sched(method meth) %{ // TODO: PPC port $archOpcode(ppc64Opcode_bl); @@ -3513,6 +3431,10 @@ encode %{ // Create a call trampoline stub for the given method. const address entry_point = !($meth$$method) ? 0 : (address)$meth$$method; const address entry_point_const = __ address_constant(entry_point, RelocationHolder::none); + if (entry_point_const == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } const int entry_point_const_toc_offset = __ offset_to_method_toc(entry_point_const); CallStubImpl::emit_trampoline_stub(_masm, entry_point_const_toc_offset, __ offset()); if (ra_->C->env()->failing()) { return; } // Code cache may be full. @@ -3530,8 +3452,8 @@ encode %{ const address virtual_call_oop_addr = __ addr_at(virtual_call_oop_addr_offset); assert(MacroAssembler::is_load_const_from_method_toc_at(virtual_call_oop_addr), "should be load from TOC"); - - __ relocate(virtual_call_Relocation::spec(virtual_call_oop_addr)); + int method_index = resolved_method_index(cbuf); + __ relocate(virtual_call_Relocation::spec(virtual_call_oop_addr, method_index)); } // At this point I do not have the address of the trampoline stub, @@ -3564,6 +3486,7 @@ encode %{ call->_jvmadj = _jvmadj; call->_in_rms = _in_rms; call->_nesting = _nesting; + call->_override_symbolic_info = _override_symbolic_info; // New call needs all inputs of old call. // Req... @@ -3620,7 +3543,11 @@ encode %{ address virtual_call_meta_addr = __ pc(); // Load a clear inline cache. AddressLiteral empty_ic((address) Universe::non_oop_word()); - __ load_const_from_method_toc(ic_reg, empty_ic, Rtoc); + bool success = __ load_const_from_method_toc(ic_reg, empty_ic, Rtoc, /*fixed_size*/ true); + if (!success) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } // CALL to fixup routine. Fixup routine uses ScopeDesc info // to determine who we intended to call. __ relocate(virtual_call_Relocation::spec(virtual_call_meta_addr)); @@ -3676,7 +3603,11 @@ encode %{ __ calculate_address_from_global_toc(Rtoc, __ method_toc()); // Put entry, env, toc into the constant pool, this needs up to 3 constant // pool entries; call_c_using_toc will optimize the call. - __ call_c_using_toc(fd, relocInfo::runtime_call_type, Rtoc); + bool success = __ call_c_using_toc(fd, relocInfo::runtime_call_type, Rtoc); + if (!success) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } #endif // Check the ret_addr_offset. @@ -6263,6 +6194,10 @@ instruct loadConF(regF dst, immF src, iRegLdst toc) %{ ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_lfs); address float_address = __ float_constant($src$$constant); + if (float_address == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } __ lfs($dst$$FloatRegister, __ offset_to_method_toc(float_address), $toc$$Register); %} ins_pipe(pipe_class_memory); @@ -6284,6 +6219,10 @@ instruct loadConFComp(regF dst, immF src, iRegLdst toc) %{ FloatRegister Rdst = $dst$$FloatRegister; Register Rtoc = $toc$$Register; address float_address = __ float_constant($src$$constant); + if (float_address == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } int offset = __ offset_to_method_toc(float_address); int hi = (offset + (1<<15))>>16; int lo = offset - hi * (1<<16); @@ -6318,7 +6257,12 @@ instruct loadConD(regD dst, immD src, iRegLdst toc) %{ size(4); ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_lfd); - int offset = __ offset_to_method_toc(__ double_constant($src$$constant)); + address float_address = __ double_constant($src$$constant); + if (float_address == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } + int offset = __ offset_to_method_toc(float_address); __ lfd($dst$$FloatRegister, offset, $toc$$Register); %} ins_pipe(pipe_class_memory); @@ -6340,7 +6284,11 @@ instruct loadConDComp(regD dst, immD src, iRegLdst toc) %{ FloatRegister Rdst = $dst$$FloatRegister; Register Rtoc = $toc$$Register; address float_address = __ double_constant($src$$constant); - int offset = __ offset_to_method_toc(float_address); + if (float_address == NULL) { + ciEnv::current()->record_out_of_memory_failure(); + return; + } + int offset = __ offset_to_method_toc(float_address); int hi = (offset + (1<<15))>>16; int lo = offset - hi * (1<<16); @@ -10949,16 +10897,16 @@ instruct partialSubtypeCheck(iRegPdst result, iRegP_N2P subklass, iRegP_N2P supe // inlined locking and unlocking -instruct cmpFastLock(flagsReg crx, iRegPdst oop, iRegPdst box, iRegPdst tmp1, iRegPdst tmp2, iRegPdst tmp3) %{ +instruct cmpFastLock(flagsReg crx, iRegPdst oop, iRegPdst box, iRegPdst tmp1, iRegPdst tmp2) %{ match(Set crx (FastLock oop box)); - effect(TEMP tmp1, TEMP tmp2, TEMP tmp3); + effect(TEMP tmp1, TEMP tmp2); predicate(!Compile::current()->use_rtm()); - format %{ "FASTLOCK $oop, $box, $tmp1, $tmp2, $tmp3" %} + format %{ "FASTLOCK $oop, $box, $tmp1, $tmp2" %} ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_compound); __ compiler_fast_lock_object($crx$$CondRegister, $oop$$Register, $box$$Register, - $tmp3$$Register, $tmp1$$Register, $tmp2$$Register, + $tmp1$$Register, $tmp2$$Register, /*tmp3*/ R0, UseBiasedLocking && !UseOptoBiasInlining); // If locking was successfull, crx should indicate 'EQ'. // The compiler generates a branch to the runtime call to @@ -10977,7 +10925,7 @@ instruct cmpFastLock_tm(flagsReg crx, iRegPdst oop, rarg2RegP box, iRegPdst tmp1 ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_compound); __ compiler_fast_lock_object($crx$$CondRegister, $oop$$Register, $box$$Register, - $tmp3$$Register, $tmp1$$Register, $tmp2$$Register, + $tmp1$$Register, $tmp2$$Register, $tmp3$$Register, /*Biased Locking*/ false, _rtm_counters, _stack_rtm_counters, ((Method*)(ra_->C->method()->constant_encoding()))->method_data(), @@ -10998,7 +10946,7 @@ instruct cmpFastUnlock(flagsReg crx, iRegPdst oop, iRegPdst box, iRegPdst tmp1, ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_compound); __ compiler_fast_unlock_object($crx$$CondRegister, $oop$$Register, $box$$Register, - $tmp3$$Register, $tmp1$$Register, $tmp2$$Register, + $tmp1$$Register, $tmp2$$Register, $tmp3$$Register, UseBiasedLocking && !UseOptoBiasInlining, false); // If unlocking was successfull, crx should indicate 'EQ'. @@ -11017,7 +10965,7 @@ instruct cmpFastUnlock_tm(flagsReg crx, iRegPdst oop, iRegPdst box, iRegPdst tmp ins_encode %{ // TODO: PPC port $archOpcode(ppc64Opcode_compound); __ compiler_fast_unlock_object($crx$$CondRegister, $oop$$Register, $box$$Register, - $tmp3$$Register, $tmp1$$Register, $tmp2$$Register, + $tmp1$$Register, $tmp2$$Register, $tmp3$$Register, /*Biased Locking*/ false, /*TM*/ true); // If unlocking was successfull, crx should indicate 'EQ'. // The compiler generates a branch to the runtime call to @@ -11790,7 +11738,6 @@ instruct safePoint_poll_conPollAddr(rscratch2RegP poll) %{ instruct CallStaticJavaDirect(method meth) %{ match(CallStaticJava); effect(USE meth); - predicate(!((CallStaticJavaNode*)n)->is_method_handle_invoke()); ins_cost(CALL_COST); ins_num_consts(3 /* up to 3 patchable constants: inline cache, 2 call targets. */); @@ -11801,20 +11748,6 @@ instruct CallStaticJavaDirect(method meth) %{ ins_pipe(pipe_class_call); %} -// Schedulable version of call static node. -instruct CallStaticJavaDirectHandle(method meth) %{ - match(CallStaticJava); - effect(USE meth); - predicate(((CallStaticJavaNode*)n)->is_method_handle_invoke()); - ins_cost(CALL_COST); - - ins_num_consts(3 /* up to 3 patchable constants: inline cache, 2 call targets. */); - - format %{ "CALL,static $meth \t// ==> " %} - ins_encode( enc_java_handle_call(meth) ); - ins_pipe(pipe_class_call); -%} - // Call Java Dynamic Instruction // Used by postalloc expand of CallDynamicJavaDirectSchedEx (actual call). diff --git a/hotspot/src/cpu/ppc/vm/register_ppc.hpp b/hotspot/src/cpu/ppc/vm/register_ppc.hpp index d97d3aea93b..def1f6e3872 100644 --- a/hotspot/src/cpu/ppc/vm/register_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/register_ppc.hpp @@ -609,11 +609,16 @@ REGISTER_DECLARATION(Register, R26_tmp6, R26); REGISTER_DECLARATION(Register, R27_tmp7, R27); REGISTER_DECLARATION(Register, R28_tmp8, R28); REGISTER_DECLARATION(Register, R29_tmp9, R29); +#ifndef CC_INTERP REGISTER_DECLARATION(Register, R24_dispatch_addr, R24); REGISTER_DECLARATION(Register, R25_templateTableBase, R25); REGISTER_DECLARATION(Register, R26_monitor, R26); REGISTER_DECLARATION(Register, R27_constPoolCache, R27); REGISTER_DECLARATION(Register, R28_mdx, R28); +#endif // CC_INTERP + +REGISTER_DECLARATION(Register, R19_inline_cache_reg, R19); +REGISTER_DECLARATION(Register, R29_TOC, R29); #ifndef DONT_USE_REGISTER_DEFINES #define R21_tmp1 AS_REGISTER(Register, R21) @@ -635,7 +640,11 @@ REGISTER_DECLARATION(Register, R28_mdx, R28); #define R28_mdx AS_REGISTER(Register, R28) #endif +#define R19_inline_cache_reg AS_REGISTER(Register, R19) +#define R29_TOC AS_REGISTER(Register, R29) + #define CCR4_is_synced AS_REGISTER(ConditionRegister, CCR4) +#endif // Scratch registers are volatile. REGISTER_DECLARATION(Register, R11_scratch1, R11); diff --git a/hotspot/src/cpu/ppc/vm/relocInfo_ppc.cpp b/hotspot/src/cpu/ppc/vm/relocInfo_ppc.cpp index 74e72ba62b6..9c5065d0dd4 100644 --- a/hotspot/src/cpu/ppc/vm/relocInfo_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/relocInfo_ppc.cpp @@ -84,13 +84,11 @@ address Relocation::pd_call_destination(address orig_addr) { NativeConditionalFarBranch* branch = NativeConditionalFarBranch_at(inst_loc); return branch->branch_destination(); } else { - // There are two instructions at the beginning of a stub, therefore we - // load at orig_addr + 8. orig_addr = nativeCall_at(inst_loc)->get_trampoline(); if (orig_addr == NULL) { return (address) -1; } else { - return (address) nativeMovConstReg_at(orig_addr + 8)->data(); + return ((NativeCallTrampolineStub*)orig_addr)->destination(); } } } diff --git a/hotspot/src/cpu/ppc/vm/runtime_ppc.cpp b/hotspot/src/cpu/ppc/vm/runtime_ppc.cpp index 404df938777..e887877d14e 100644 --- a/hotspot/src/cpu/ppc/vm/runtime_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/runtime_ppc.cpp @@ -1,6 +1,6 @@ /* - * Copyright (c) 1998, 2014, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012, 2014 SAP AG. All rights reserved. + * Copyright (c) 1998, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012, 2015 SAP AG. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -45,16 +45,6 @@ #ifdef COMPILER2 -// SP adjustment (must use unextended SP) for method handle call sites -// during exception handling. -static intptr_t adjust_SP_for_methodhandle_callsite(JavaThread *thread) { - RegisterMap map(thread, false); - // The frame constructor will do the correction for us (see frame::adjust_unextended_SP). - frame mh_caller_frame = thread->last_frame().sender(&map); - assert(mh_caller_frame.is_compiled_frame(), "Only may reach here for compiled MH call sites"); - return (intptr_t) mh_caller_frame.unextended_sp(); -} - //------------------------------generate_exception_blob--------------------------- // Creates exception blob at the end. // Using exception blob, this code is jumped from a compiled method. @@ -129,17 +119,10 @@ void OptoRuntime::generate_exception_blob() { OopMapSet* oop_maps = new OopMapSet(); oop_maps->add_gc_map(calls_return_pc - start, map); - // Get unextended_sp for method handle call sites. - Label mh_callsite, mh_done; // Use a 2nd c call if it's a method handle call site. - __ lwa(R4_ARG2, in_bytes(JavaThread::is_method_handle_return_offset()), R16_thread); - __ cmpwi(CCR0, R4_ARG2, 0); - __ bne(CCR0, mh_callsite); - __ mtctr(R3_RET); // Move address of exception handler to SR_CTR. __ reset_last_Java_frame(); __ pop_frame(); - __ bind(mh_done); // We have a handler in register SR_CTR (could be deopt blob). // Get the exception oop. @@ -161,25 +144,6 @@ void OptoRuntime::generate_exception_blob() { __ mtlr(R4_ARG2); __ bctr(); - - // Same as above, but also set sp to unextended_sp. - __ bind(mh_callsite); - __ mr(R31, R3_RET); // Save branch address. - __ mr(R3_ARG1, R16_thread); -#if defined(ABI_ELFv2) - __ call_c((address) adjust_SP_for_methodhandle_callsite, relocInfo::none); -#else - __ call_c(CAST_FROM_FN_PTR(FunctionDescriptor*, adjust_SP_for_methodhandle_callsite), relocInfo::none); -#endif - // Returns unextended_sp in R3_RET. - - __ mtctr(R31); // Move address of exception handler to SR_CTR. - __ reset_last_Java_frame(); - - __ mr(R1_SP, R3_RET); // Set sp to unextended_sp. - __ b(mh_done); - - // Make sure all code is generated. masm->flush(); diff --git a/hotspot/src/cpu/ppc/vm/sharedRuntime_ppc.cpp b/hotspot/src/cpu/ppc/vm/sharedRuntime_ppc.cpp index d6642b5ca9b..5e7c7c2a0a4 100644 --- a/hotspot/src/cpu/ppc/vm/sharedRuntime_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/sharedRuntime_ppc.cpp @@ -44,6 +44,8 @@ #include "opto/runtime.hpp" #endif +#include + #define __ masm-> #ifdef PRODUCT @@ -62,7 +64,7 @@ class RegisterSaver { // Support different return pc locations. enum ReturnPCLocation { return_pc_is_lr, - return_pc_is_r4, + return_pc_is_pre_saved, return_pc_is_thread_saved_exception_pc }; @@ -241,16 +243,17 @@ OopMap* RegisterSaver::push_frame_reg_args_and_save_live_registers(MacroAssemble __ mfcr(R31); __ std(R31, _abi(cr), R1_SP); switch (return_pc_location) { - case return_pc_is_lr: __ mflr(R31); break; - case return_pc_is_r4: __ mr(R31, R4); break; - case return_pc_is_thread_saved_exception_pc: - __ ld(R31, thread_(saved_exception_pc)); break; + case return_pc_is_lr: __ mflr(R31); break; + case return_pc_is_pre_saved: assert(return_pc_adjustment == 0, "unsupported"); break; + case return_pc_is_thread_saved_exception_pc: __ ld(R31, thread_(saved_exception_pc)); break; default: ShouldNotReachHere(); } - if (return_pc_adjustment != 0) { - __ addi(R31, R31, return_pc_adjustment); + if (return_pc_location != return_pc_is_pre_saved) { + if (return_pc_adjustment != 0) { + __ addi(R31, R31, return_pc_adjustment); + } + __ std(R31, _abi(lr), R1_SP); } - __ std(R31, _abi(lr), R1_SP); // push a new frame __ push_frame(frame_size_in_bytes, R31); @@ -646,7 +649,7 @@ int SharedRuntime::java_calling_convention(const BasicType *sig_bt, return round_to(stk, 2); } -#ifdef COMPILER2 +#if defined(COMPILER1) || defined(COMPILER2) // Calling convention for calling C code. int SharedRuntime::c_calling_convention(const BasicType *sig_bt, VMRegPair *regs, @@ -2566,7 +2569,7 @@ uint SharedRuntime::out_preserve_stack_slots() { #endif } -#ifdef COMPILER2 +#if defined(COMPILER1) || defined(COMPILER2) // Frame generation for deopt and uncommon trap blobs. static void push_skeleton_frame(MacroAssembler* masm, bool deopt, /* Read */ @@ -2713,7 +2716,7 @@ void SharedRuntime::generate_deopt_blob() { const address start = __ pc(); -#ifdef COMPILER2 +#if defined(COMPILER1) || defined(COMPILER2) // -------------------------------------------------------------------------- // Prolog for non exception case! @@ -2762,28 +2765,43 @@ void SharedRuntime::generate_deopt_blob() { BLOCK_COMMENT("Prolog for exception case"); - // The RegisterSaves doesn't need to adjust the return pc for this situation. - const int return_pc_adjustment_exception = 0; - - // Push the "unpack frame". - // Save everything in sight. - assert(R4 == R4_ARG2, "exception pc must be in r4"); - RegisterSaver::push_frame_reg_args_and_save_live_registers(masm, - &first_frame_size_in_bytes, - /*generate_oop_map=*/ false, - return_pc_adjustment_exception, - RegisterSaver::return_pc_is_r4); - - // Deopt during an exception. Save exec mode for unpack_frames. - __ li(exec_mode_reg, Deoptimization::Unpack_exception); - // Store exception oop and pc in thread (location known to GC). // This is needed since the call to "fetch_unroll_info()" may safepoint. __ std(R3_ARG1, in_bytes(JavaThread::exception_oop_offset()), R16_thread); __ std(R4_ARG2, in_bytes(JavaThread::exception_pc_offset()), R16_thread); + __ std(R4_ARG2, _abi(lr), R1_SP); + + // Vanilla deoptimization with an exception pending in exception_oop. + int exception_in_tls_offset = __ pc() - start; + + // Push the "unpack frame". + // Save everything in sight. + RegisterSaver::push_frame_reg_args_and_save_live_registers(masm, + &first_frame_size_in_bytes, + /*generate_oop_map=*/ false, + /*return_pc_adjustment_exception=*/ 0, + RegisterSaver::return_pc_is_pre_saved); + + // Deopt during an exception. Save exec mode for unpack_frames. + __ li(exec_mode_reg, Deoptimization::Unpack_exception); // fall through + int reexecute_offset = 0; +#ifdef COMPILER1 + __ b(exec_mode_initialized); + + // Reexecute entry, similar to c2 uncommon trap + reexecute_offset = __ pc() - start; + + RegisterSaver::push_frame_reg_args_and_save_live_registers(masm, + &first_frame_size_in_bytes, + /*generate_oop_map=*/ false, + /*return_pc_adjustment_reexecute=*/ 0, + RegisterSaver::return_pc_is_pre_saved); + __ li(exec_mode_reg, Deoptimization::Unpack_reexecute); +#endif + // -------------------------------------------------------------------------- __ BIND(exec_mode_initialized); @@ -2889,7 +2907,9 @@ void SharedRuntime::generate_deopt_blob() { int exception_offset = __ pc() - start; #endif // COMPILER2 - _deopt_blob = DeoptimizationBlob::create(&buffer, oop_maps, 0, exception_offset, 0, first_frame_size_in_bytes / wordSize); + _deopt_blob = DeoptimizationBlob::create(&buffer, oop_maps, 0, exception_offset, + reexecute_offset, first_frame_size_in_bytes / wordSize); + _deopt_blob->set_unpack_with_exception_in_tls_offset(exception_in_tls_offset); } #ifdef COMPILER2 @@ -3196,3 +3216,245 @@ RuntimeStub* SharedRuntime::generate_resolve_blob(address destination, const cha return RuntimeStub::new_runtime_stub(name, &buffer, frame_complete, frame_size_in_bytes/wordSize, oop_maps, true); } + + +//------------------------------Montgomery multiplication------------------------ +// + +// Subtract 0:b from carry:a. Return carry. +static unsigned long +sub(unsigned long a[], unsigned long b[], unsigned long carry, long len) { + long i = 0; + unsigned long tmp, tmp2; + __asm__ __volatile__ ( + "subfc %[tmp], %[tmp], %[tmp] \n" // pre-set CA + "mtctr %[len] \n" + "0: \n" + "ldx %[tmp], %[i], %[a] \n" + "ldx %[tmp2], %[i], %[b] \n" + "subfe %[tmp], %[tmp2], %[tmp] \n" // subtract extended + "stdx %[tmp], %[i], %[a] \n" + "addi %[i], %[i], 8 \n" + "bdnz 0b \n" + "addme %[tmp], %[carry] \n" // carry + CA - 1 + : [i]"+b"(i), [tmp]"=&r"(tmp), [tmp2]"=&r"(tmp2) + : [a]"r"(a), [b]"r"(b), [carry]"r"(carry), [len]"r"(len) + : "ctr", "xer", "memory" + ); + return tmp; +} + +// Multiply (unsigned) Long A by Long B, accumulating the double- +// length result into the accumulator formed of T0, T1, and T2. +inline void MACC(unsigned long A, unsigned long B, unsigned long &T0, unsigned long &T1, unsigned long &T2) { + unsigned long hi, lo; + __asm__ __volatile__ ( + "mulld %[lo], %[A], %[B] \n" + "mulhdu %[hi], %[A], %[B] \n" + "addc %[T0], %[T0], %[lo] \n" + "adde %[T1], %[T1], %[hi] \n" + "addze %[T2], %[T2] \n" + : [hi]"=&r"(hi), [lo]"=&r"(lo), [T0]"+r"(T0), [T1]"+r"(T1), [T2]"+r"(T2) + : [A]"r"(A), [B]"r"(B) + : "xer" + ); +} + +// As above, but add twice the double-length result into the +// accumulator. +inline void MACC2(unsigned long A, unsigned long B, unsigned long &T0, unsigned long &T1, unsigned long &T2) { + unsigned long hi, lo; + __asm__ __volatile__ ( + "mulld %[lo], %[A], %[B] \n" + "mulhdu %[hi], %[A], %[B] \n" + "addc %[T0], %[T0], %[lo] \n" + "adde %[T1], %[T1], %[hi] \n" + "addze %[T2], %[T2] \n" + "addc %[T0], %[T0], %[lo] \n" + "adde %[T1], %[T1], %[hi] \n" + "addze %[T2], %[T2] \n" + : [hi]"=&r"(hi), [lo]"=&r"(lo), [T0]"+r"(T0), [T1]"+r"(T1), [T2]"+r"(T2) + : [A]"r"(A), [B]"r"(B) + : "xer" + ); +} + +// Fast Montgomery multiplication. The derivation of the algorithm is +// in "A Cryptographic Library for the Motorola DSP56000, +// Dusse and Kaliski, Proc. EUROCRYPT 90, pp. 230-237". +static void +montgomery_multiply(unsigned long a[], unsigned long b[], unsigned long n[], + unsigned long m[], unsigned long inv, int len) { + unsigned long t0 = 0, t1 = 0, t2 = 0; // Triple-precision accumulator + int i; + + assert(inv * n[0] == -1UL, "broken inverse in Montgomery multiply"); + + for (i = 0; i < len; i++) { + int j; + for (j = 0; j < i; j++) { + MACC(a[j], b[i-j], t0, t1, t2); + MACC(m[j], n[i-j], t0, t1, t2); + } + MACC(a[i], b[0], t0, t1, t2); + m[i] = t0 * inv; + MACC(m[i], n[0], t0, t1, t2); + + assert(t0 == 0, "broken Montgomery multiply"); + + t0 = t1; t1 = t2; t2 = 0; + } + + for (i = len; i < 2*len; i++) { + int j; + for (j = i-len+1; j < len; j++) { + MACC(a[j], b[i-j], t0, t1, t2); + MACC(m[j], n[i-j], t0, t1, t2); + } + m[i-len] = t0; + t0 = t1; t1 = t2; t2 = 0; + } + + while (t0) { + t0 = sub(m, n, t0, len); + } +} + +// Fast Montgomery squaring. This uses asymptotically 25% fewer +// multiplies so it should be up to 25% faster than Montgomery +// multiplication. However, its loop control is more complex and it +// may actually run slower on some machines. +static void +montgomery_square(unsigned long a[], unsigned long n[], + unsigned long m[], unsigned long inv, int len) { + unsigned long t0 = 0, t1 = 0, t2 = 0; // Triple-precision accumulator + int i; + + assert(inv * n[0] == -1UL, "broken inverse in Montgomery multiply"); + + for (i = 0; i < len; i++) { + int j; + int end = (i+1)/2; + for (j = 0; j < end; j++) { + MACC2(a[j], a[i-j], t0, t1, t2); + MACC(m[j], n[i-j], t0, t1, t2); + } + if ((i & 1) == 0) { + MACC(a[j], a[j], t0, t1, t2); + } + for (; j < i; j++) { + MACC(m[j], n[i-j], t0, t1, t2); + } + m[i] = t0 * inv; + MACC(m[i], n[0], t0, t1, t2); + + assert(t0 == 0, "broken Montgomery square"); + + t0 = t1; t1 = t2; t2 = 0; + } + + for (i = len; i < 2*len; i++) { + int start = i-len+1; + int end = start + (len - start)/2; + int j; + for (j = start; j < end; j++) { + MACC2(a[j], a[i-j], t0, t1, t2); + MACC(m[j], n[i-j], t0, t1, t2); + } + if ((i & 1) == 0) { + MACC(a[j], a[j], t0, t1, t2); + } + for (; j < len; j++) { + MACC(m[j], n[i-j], t0, t1, t2); + } + m[i-len] = t0; + t0 = t1; t1 = t2; t2 = 0; + } + + while (t0) { + t0 = sub(m, n, t0, len); + } +} + +// The threshold at which squaring is advantageous was determined +// experimentally on an i7-3930K (Ivy Bridge) CPU @ 3.5GHz. +// Doesn't seem to be relevant for Power8 so we use the same value. +#define MONTGOMERY_SQUARING_THRESHOLD 64 + +// Copy len longwords from s to d, word-swapping as we go. The +// destination array is reversed. +static void reverse_words(unsigned long *s, unsigned long *d, int len) { + d += len; + while(len-- > 0) { + d--; + unsigned long s_val = *s; + // Swap words in a longword on little endian machines. +#ifdef VM_LITTLE_ENDIAN + s_val = (s_val << 32) | (s_val >> 32); +#endif + *d = s_val; + s++; + } +} + +void SharedRuntime::montgomery_multiply(jint *a_ints, jint *b_ints, jint *n_ints, + jint len, jlong inv, + jint *m_ints) { + assert(len % 2 == 0, "array length in montgomery_multiply must be even"); + int longwords = len/2; + assert(longwords > 0, "unsupported"); + + // Make very sure we don't use so much space that the stack might + // overflow. 512 jints corresponds to an 16384-bit integer and + // will use here a total of 8k bytes of stack space. + int total_allocation = longwords * sizeof (unsigned long) * 4; + guarantee(total_allocation <= 8192, "must be"); + unsigned long *scratch = (unsigned long *)alloca(total_allocation); + + // Local scratch arrays + unsigned long + *a = scratch + 0 * longwords, + *b = scratch + 1 * longwords, + *n = scratch + 2 * longwords, + *m = scratch + 3 * longwords; + + reverse_words((unsigned long *)a_ints, a, longwords); + reverse_words((unsigned long *)b_ints, b, longwords); + reverse_words((unsigned long *)n_ints, n, longwords); + + ::montgomery_multiply(a, b, n, m, (unsigned long)inv, longwords); + + reverse_words(m, (unsigned long *)m_ints, longwords); +} + +void SharedRuntime::montgomery_square(jint *a_ints, jint *n_ints, + jint len, jlong inv, + jint *m_ints) { + assert(len % 2 == 0, "array length in montgomery_square must be even"); + int longwords = len/2; + assert(longwords > 0, "unsupported"); + + // Make very sure we don't use so much space that the stack might + // overflow. 512 jints corresponds to an 16384-bit integer and + // will use here a total of 6k bytes of stack space. + int total_allocation = longwords * sizeof (unsigned long) * 3; + guarantee(total_allocation <= 8192, "must be"); + unsigned long *scratch = (unsigned long *)alloca(total_allocation); + + // Local scratch arrays + unsigned long + *a = scratch + 0 * longwords, + *n = scratch + 1 * longwords, + *m = scratch + 2 * longwords; + + reverse_words((unsigned long *)a_ints, a, longwords); + reverse_words((unsigned long *)n_ints, n, longwords); + + if (len >= MONTGOMERY_SQUARING_THRESHOLD) { + ::montgomery_square(a, n, m, (unsigned long)inv, longwords); + } else { + ::montgomery_multiply(a, a, n, m, (unsigned long)inv, longwords); + } + + reverse_words(m, (unsigned long *)m_ints, longwords); +} diff --git a/hotspot/src/cpu/ppc/vm/stubGenerator_ppc.cpp b/hotspot/src/cpu/ppc/vm/stubGenerator_ppc.cpp index 1a4493d96a0..78082716d85 100644 --- a/hotspot/src/cpu/ppc/vm/stubGenerator_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/stubGenerator_ppc.cpp @@ -48,6 +48,12 @@ #define BLOCK_COMMENT(str) __ block_comment(str) #endif +#if defined(ABI_ELFv2) +#define STUB_ENTRY(name) StubRoutines::name() +#else +#define STUB_ENTRY(name) ((FunctionDescriptor*)StubRoutines::name())->entry() +#endif + class StubGenerator: public StubCodeGenerator { private: @@ -252,8 +258,7 @@ class StubGenerator: public StubCodeGenerator { // // global toc register - __ load_const(R29, MacroAssembler::global_toc(), R11_scratch1); - + __ load_const_optimized(R29_TOC, MacroAssembler::global_toc(), R11_scratch1); // Remember the senderSP so we interpreter can pop c2i arguments off of the stack // when called via a c2i. @@ -612,14 +617,17 @@ class StubGenerator: public StubCodeGenerator { // Kills: // nothing // - void gen_write_ref_array_pre_barrier(Register from, Register to, Register count, bool dest_uninitialized, Register Rtmp1) { + void gen_write_ref_array_pre_barrier(Register from, Register to, Register count, bool dest_uninitialized, Register Rtmp1, + Register preserve1 = noreg, Register preserve2 = noreg) { BarrierSet* const bs = Universe::heap()->barrier_set(); switch (bs->kind()) { case BarrierSet::G1SATBCTLogging: // With G1, don't generate the call if we statically know that the target in uninitialized if (!dest_uninitialized) { - const int spill_slots = 4 * wordSize; - const int frame_size = frame::abi_reg_args_size + spill_slots; + int spill_slots = 3; + if (preserve1 != noreg) { spill_slots++; } + if (preserve2 != noreg) { spill_slots++; } + const int frame_size = align_size_up(frame::abi_reg_args_size + spill_slots * BytesPerWord, frame::alignment_in_bytes); Label filtered; // Is marking active? @@ -633,17 +641,23 @@ class StubGenerator: public StubCodeGenerator { __ beq(CCR0, filtered); __ save_LR_CR(R0); - __ push_frame_reg_args(spill_slots, R0); - __ std(from, frame_size - 1 * wordSize, R1_SP); - __ std(to, frame_size - 2 * wordSize, R1_SP); - __ std(count, frame_size - 3 * wordSize, R1_SP); + __ push_frame(frame_size, R0); + int slot_nr = 0; + __ std(from, frame_size - (++slot_nr) * wordSize, R1_SP); + __ std(to, frame_size - (++slot_nr) * wordSize, R1_SP); + __ std(count, frame_size - (++slot_nr) * wordSize, R1_SP); + if (preserve1 != noreg) { __ std(preserve1, frame_size - (++slot_nr) * wordSize, R1_SP); } + if (preserve2 != noreg) { __ std(preserve2, frame_size - (++slot_nr) * wordSize, R1_SP); } __ call_VM_leaf(CAST_FROM_FN_PTR(address, BarrierSet::static_write_ref_array_pre), to, count); - __ ld(from, frame_size - 1 * wordSize, R1_SP); - __ ld(to, frame_size - 2 * wordSize, R1_SP); - __ ld(count, frame_size - 3 * wordSize, R1_SP); - __ pop_frame(); + slot_nr = 0; + __ ld(from, frame_size - (++slot_nr) * wordSize, R1_SP); + __ ld(to, frame_size - (++slot_nr) * wordSize, R1_SP); + __ ld(count, frame_size - (++slot_nr) * wordSize, R1_SP); + if (preserve1 != noreg) { __ ld(preserve1, frame_size - (++slot_nr) * wordSize, R1_SP); } + if (preserve2 != noreg) { __ ld(preserve2, frame_size - (++slot_nr) * wordSize, R1_SP); } + __ addi(R1_SP, R1_SP, frame_size); // pop_frame() __ restore_LR_CR(R0); __ bind(filtered); @@ -667,27 +681,22 @@ class StubGenerator: public StubCodeGenerator { // // The input registers and R0 are overwritten. // - void gen_write_ref_array_post_barrier(Register addr, Register count, Register tmp, bool branchToEnd) { + void gen_write_ref_array_post_barrier(Register addr, Register count, Register tmp, Register preserve = noreg) { BarrierSet* const bs = Universe::heap()->barrier_set(); switch (bs->kind()) { case BarrierSet::G1SATBCTLogging: { - if (branchToEnd) { - __ save_LR_CR(R0); - // We need this frame only to spill LR. - __ push_frame_reg_args(0, R0); - __ call_VM_leaf(CAST_FROM_FN_PTR(address, BarrierSet::static_write_ref_array_post), addr, count); - __ pop_frame(); - __ restore_LR_CR(R0); - } else { - // Tail call: fake call from stub caller by branching without linking. - address entry_point = (address)CAST_FROM_FN_PTR(address, BarrierSet::static_write_ref_array_post); - __ mr_if_needed(R3_ARG1, addr); - __ mr_if_needed(R4_ARG2, count); - __ load_const(R11, entry_point, R0); - __ call_c_and_return_to_caller(R11); - } + int spill_slots = (preserve != noreg) ? 1 : 0; + const int frame_size = align_size_up(frame::abi_reg_args_size + spill_slots * BytesPerWord, frame::alignment_in_bytes); + + __ save_LR_CR(R0); + __ push_frame(frame_size, R0); + if (preserve != noreg) { __ std(preserve, frame_size - 1 * wordSize, R1_SP); } + __ call_VM_leaf(CAST_FROM_FN_PTR(address, BarrierSet::static_write_ref_array_post), addr, count); + if (preserve != noreg) { __ ld(preserve, frame_size - 1 * wordSize, R1_SP); } + __ addi(R1_SP, R1_SP, frame_size); // pop_frame(); + __ restore_LR_CR(R0); } break; case BarrierSet::CardTableForRS: @@ -722,12 +731,9 @@ class StubGenerator: public StubCodeGenerator { __ addi(addr, addr, 1); __ bdnz(Lstore_loop); __ bind(Lskip_loop); - - if (!branchToEnd) __ blr(); } break; case BarrierSet::ModRef: - if (!branchToEnd) __ blr(); break; default: ShouldNotReachHere(); @@ -756,8 +762,10 @@ class StubGenerator: public StubCodeGenerator { // Procedure for large arrays (uses data cache block zero instruction). Label dwloop, fast, fastloop, restloop, lastdword, done; - int cl_size=VM_Version::get_cache_line_size(), cl_dwords=cl_size>>3, cl_dwordaddr_bits=exact_log2(cl_dwords); - int min_dcbz=2; // Needs to be positive, apply dcbz only to at least min_dcbz cache lines. + int cl_size = VM_Version::L1_data_cache_line_size(); + int cl_dwords = cl_size >> 3; + int cl_dwordaddr_bits = exact_log2(cl_dwords); + int min_dcbz = 2; // Needs to be positive, apply dcbz only to at least min_dcbz cache lines. // Clear up to 128byte boundary if long enough, dword_cnt=(16-(base>>3))%16. __ dcbtst(base_ptr_reg); // Indicate write access to first cache line ... @@ -1074,7 +1082,6 @@ class StubGenerator: public StubCodeGenerator { Register tmp1 = R6_ARG4; Register tmp2 = R7_ARG5; - Label l_overlap; #ifdef ASSERT __ srdi_(tmp2, R5_ARG3, 31); __ asm_assert_eq("missing zero extend", 0xAFFE); @@ -1084,19 +1091,11 @@ class StubGenerator: public StubCodeGenerator { __ sldi(tmp2, R5_ARG3, log2_elem_size); // size in bytes __ cmpld(CCR0, R3_ARG1, R4_ARG2); // Use unsigned comparison! __ cmpld(CCR1, tmp1, tmp2); - __ crand(CCR0, Assembler::less, CCR1, Assembler::less); - __ blt(CCR0, l_overlap); // Src before dst and distance smaller than size. + __ crnand(CCR0, Assembler::less, CCR1, Assembler::less); + // Overlaps if Src before dst and distance smaller than size. + // Branch to forward copy routine otherwise (within range of 32kB). + __ bc(Assembler::bcondCRbiIs1, Assembler::bi0(CCR0, Assembler::less), no_overlap_target); - // need to copy forwards - if (__ is_within_range_of_b(no_overlap_target, __ pc())) { - __ b(no_overlap_target); - } else { - __ load_const(tmp1, no_overlap_target, tmp2); - __ mtctr(tmp1); - __ bctr(); - } - - __ bind(l_overlap); // need to copy backwards } @@ -1241,6 +1240,7 @@ class StubGenerator: public StubCodeGenerator { } __ bind(l_4); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1262,15 +1262,9 @@ class StubGenerator: public StubCodeGenerator { Register tmp2 = R7_ARG5; Register tmp3 = R8_ARG6; -#if defined(ABI_ELFv2) address nooverlap_target = aligned ? - StubRoutines::arrayof_jbyte_disjoint_arraycopy() : - StubRoutines::jbyte_disjoint_arraycopy(); -#else - address nooverlap_target = aligned ? - ((FunctionDescriptor*)StubRoutines::arrayof_jbyte_disjoint_arraycopy())->entry() : - ((FunctionDescriptor*)StubRoutines::jbyte_disjoint_arraycopy())->entry(); -#endif + STUB_ENTRY(arrayof_jbyte_disjoint_arraycopy) : + STUB_ENTRY(jbyte_disjoint_arraycopy); array_overlap_test(nooverlap_target, 0); // Do reverse copy. We assume the case of actual overlap is rare enough @@ -1285,6 +1279,7 @@ class StubGenerator: public StubCodeGenerator { __ lbzx(tmp1, R3_ARG1, R5_ARG3); __ bge(CCR0, l_1); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1467,6 +1462,7 @@ class StubGenerator: public StubCodeGenerator { __ bdnz(l_5); } __ bind(l_4); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1488,15 +1484,9 @@ class StubGenerator: public StubCodeGenerator { Register tmp2 = R7_ARG5; Register tmp3 = R8_ARG6; -#if defined(ABI_ELFv2) address nooverlap_target = aligned ? - StubRoutines::arrayof_jshort_disjoint_arraycopy() : - StubRoutines::jshort_disjoint_arraycopy(); -#else - address nooverlap_target = aligned ? - ((FunctionDescriptor*)StubRoutines::arrayof_jshort_disjoint_arraycopy())->entry() : - ((FunctionDescriptor*)StubRoutines::jshort_disjoint_arraycopy())->entry(); -#endif + STUB_ENTRY(arrayof_jshort_disjoint_arraycopy) : + STUB_ENTRY(jshort_disjoint_arraycopy); array_overlap_test(nooverlap_target, 1); @@ -1510,6 +1500,7 @@ class StubGenerator: public StubCodeGenerator { __ lhzx(tmp2, R3_ARG1, tmp1); __ bge(CCR0, l_1); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1613,6 +1604,7 @@ class StubGenerator: public StubCodeGenerator { StubCodeMark mark(this, "StubRoutines", name); address start = __ function_entry(); generate_disjoint_int_copy_core(aligned); + __ li(R3_RET, 0); // return 0 __ blr(); return start; } @@ -1697,20 +1689,15 @@ class StubGenerator: public StubCodeGenerator { StubCodeMark mark(this, "StubRoutines", name); address start = __ function_entry(); -#if defined(ABI_ELFv2) address nooverlap_target = aligned ? - StubRoutines::arrayof_jint_disjoint_arraycopy() : - StubRoutines::jint_disjoint_arraycopy(); -#else - address nooverlap_target = aligned ? - ((FunctionDescriptor*)StubRoutines::arrayof_jint_disjoint_arraycopy())->entry() : - ((FunctionDescriptor*)StubRoutines::jint_disjoint_arraycopy())->entry(); -#endif + STUB_ENTRY(arrayof_jint_disjoint_arraycopy) : + STUB_ENTRY(jint_disjoint_arraycopy); array_overlap_test(nooverlap_target, 2); generate_conjoint_int_copy_core(aligned); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1789,6 +1776,7 @@ class StubGenerator: public StubCodeGenerator { StubCodeMark mark(this, "StubRoutines", name); address start = __ function_entry(); generate_disjoint_long_copy_core(aligned); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1871,19 +1859,14 @@ class StubGenerator: public StubCodeGenerator { StubCodeMark mark(this, "StubRoutines", name); address start = __ function_entry(); -#if defined(ABI_ELFv2) address nooverlap_target = aligned ? - StubRoutines::arrayof_jlong_disjoint_arraycopy() : - StubRoutines::jlong_disjoint_arraycopy(); -#else - address nooverlap_target = aligned ? - ((FunctionDescriptor*)StubRoutines::arrayof_jlong_disjoint_arraycopy())->entry() : - ((FunctionDescriptor*)StubRoutines::jlong_disjoint_arraycopy())->entry(); -#endif + STUB_ENTRY(arrayof_jlong_disjoint_arraycopy) : + STUB_ENTRY(jlong_disjoint_arraycopy); array_overlap_test(nooverlap_target, 3); generate_conjoint_long_copy_core(aligned); + __ li(R3_RET, 0); // return 0 __ blr(); return start; @@ -1903,15 +1886,9 @@ class StubGenerator: public StubCodeGenerator { address start = __ function_entry(); -#if defined(ABI_ELFv2) address nooverlap_target = aligned ? - StubRoutines::arrayof_oop_disjoint_arraycopy() : - StubRoutines::oop_disjoint_arraycopy(); -#else - address nooverlap_target = aligned ? - ((FunctionDescriptor*)StubRoutines::arrayof_oop_disjoint_arraycopy())->entry() : - ((FunctionDescriptor*)StubRoutines::oop_disjoint_arraycopy())->entry(); -#endif + STUB_ENTRY(arrayof_oop_disjoint_arraycopy) : + STUB_ENTRY(oop_disjoint_arraycopy); gen_write_ref_array_pre_barrier(R3_ARG1, R4_ARG2, R5_ARG3, dest_uninitialized, R9_ARG7); @@ -1927,7 +1904,9 @@ class StubGenerator: public StubCodeGenerator { generate_conjoint_long_copy_core(aligned); } - gen_write_ref_array_post_barrier(R9_ARG7, R10_ARG8, R11_scratch1, /*branchToEnd*/ false); + gen_write_ref_array_post_barrier(R9_ARG7, R10_ARG8, R11_scratch1); + __ li(R3_RET, 0); // return 0 + __ blr(); return start; } @@ -1957,11 +1936,460 @@ class StubGenerator: public StubCodeGenerator { generate_disjoint_long_copy_core(aligned); } - gen_write_ref_array_post_barrier(R9_ARG7, R10_ARG8, R11_scratch1, /*branchToEnd*/ false); + gen_write_ref_array_post_barrier(R9_ARG7, R10_ARG8, R11_scratch1); + __ li(R3_RET, 0); // return 0 + __ blr(); return start; } + + // Helper for generating a dynamic type check. + // Smashes only the given temp registers. + void generate_type_check(Register sub_klass, + Register super_check_offset, + Register super_klass, + Register temp, + Label& L_success) { + assert_different_registers(sub_klass, super_check_offset, super_klass); + + BLOCK_COMMENT("type_check:"); + + Label L_miss; + + __ check_klass_subtype_fast_path(sub_klass, super_klass, temp, R0, &L_success, &L_miss, NULL, + super_check_offset); + __ check_klass_subtype_slow_path(sub_klass, super_klass, temp, R0, &L_success, NULL); + + // Fall through on failure! + __ bind(L_miss); + } + + + // Generate stub for checked oop copy. + // + // Arguments for generated stub: + // from: R3 + // to: R4 + // count: R5 treated as signed + // ckoff: R6 (super_check_offset) + // ckval: R7 (super_klass) + // ret: R3 zero for success; (-1^K) where K is partial transfer count + // + address generate_checkcast_copy(const char *name, bool dest_uninitialized) { + + const Register R3_from = R3_ARG1; // source array address + const Register R4_to = R4_ARG2; // destination array address + const Register R5_count = R5_ARG3; // elements count + const Register R6_ckoff = R6_ARG4; // super_check_offset + const Register R7_ckval = R7_ARG5; // super_klass + + const Register R8_offset = R8_ARG6; // loop var, with stride wordSize + const Register R9_remain = R9_ARG7; // loop var, with stride -1 + const Register R10_oop = R10_ARG8; // actual oop copied + const Register R11_klass = R11_scratch1; // oop._klass + const Register R12_tmp = R12_scratch2; + + const Register R2_minus1 = R2; + + //__ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", name); + address start = __ function_entry(); + + // TODO: Assert that int is 64 bit sign extended and arrays are not conjoint. + + gen_write_ref_array_pre_barrier(R3_from, R4_to, R5_count, dest_uninitialized, R12_tmp, /* preserve: */ R6_ckoff, R7_ckval); + + //inc_counter_np(SharedRuntime::_checkcast_array_copy_ctr, R12_tmp, R3_RET); + + Label load_element, store_element, store_null, success, do_card_marks; + __ or_(R9_remain, R5_count, R5_count); // Initialize loop index, and test it. + __ li(R8_offset, 0); // Offset from start of arrays. + __ li(R2_minus1, -1); + __ bne(CCR0, load_element); + + // Empty array: Nothing to do. + __ li(R3_RET, 0); // Return 0 on (trivial) success. + __ blr(); + + // ======== begin loop ======== + // (Entry is load_element.) + __ align(OptoLoopAlignment); + __ bind(store_element); + if (UseCompressedOops) { + __ encode_heap_oop_not_null(R10_oop); + __ bind(store_null); + __ stw(R10_oop, R8_offset, R4_to); + } else { + __ bind(store_null); + __ std(R10_oop, R8_offset, R4_to); + } + + __ addi(R8_offset, R8_offset, heapOopSize); // Step to next offset. + __ add_(R9_remain, R2_minus1, R9_remain); // Decrement the count. + __ beq(CCR0, success); + + // ======== loop entry is here ======== + __ bind(load_element); + __ load_heap_oop(R10_oop, R8_offset, R3_from, &store_null); // Load the oop. + + __ load_klass(R11_klass, R10_oop); // Query the object klass. + + generate_type_check(R11_klass, R6_ckoff, R7_ckval, R12_tmp, + // Branch to this on success: + store_element); + // ======== end loop ======== + + // It was a real error; we must depend on the caller to finish the job. + // Register R9_remain has number of *remaining* oops, R5_count number of *total* oops. + // Emit GC store barriers for the oops we have copied (R5_count minus R9_remain), + // and report their number to the caller. + __ subf_(R5_count, R9_remain, R5_count); + __ nand(R3_RET, R5_count, R5_count); // report (-1^K) to caller + __ bne(CCR0, do_card_marks); + __ blr(); + + __ bind(success); + __ li(R3_RET, 0); + + __ bind(do_card_marks); + // Store check on R4_to[0..R5_count-1]. + gen_write_ref_array_post_barrier(R4_to, R5_count, R12_tmp, /* preserve: */ R3_RET); + __ blr(); + return start; + } + + + // Generate 'unsafe' array copy stub. + // Though just as safe as the other stubs, it takes an unscaled + // size_t argument instead of an element count. + // + // Arguments for generated stub: + // from: R3 + // to: R4 + // count: R5 byte count, treated as ssize_t, can be zero + // + // Examines the alignment of the operands and dispatches + // to a long, int, short, or byte copy loop. + // + address generate_unsafe_copy(const char* name, + address byte_copy_entry, + address short_copy_entry, + address int_copy_entry, + address long_copy_entry) { + + const Register R3_from = R3_ARG1; // source array address + const Register R4_to = R4_ARG2; // destination array address + const Register R5_count = R5_ARG3; // elements count (as long on PPC64) + + const Register R6_bits = R6_ARG4; // test copy of low bits + const Register R7_tmp = R7_ARG5; + + //__ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", name); + address start = __ function_entry(); + + // Bump this on entry, not on exit: + //inc_counter_np(SharedRuntime::_unsafe_array_copy_ctr, R6_bits, R7_tmp); + + Label short_copy, int_copy, long_copy; + + __ orr(R6_bits, R3_from, R4_to); + __ orr(R6_bits, R6_bits, R5_count); + __ andi_(R0, R6_bits, (BytesPerLong-1)); + __ beq(CCR0, long_copy); + + __ andi_(R0, R6_bits, (BytesPerInt-1)); + __ beq(CCR0, int_copy); + + __ andi_(R0, R6_bits, (BytesPerShort-1)); + __ beq(CCR0, short_copy); + + // byte_copy: + __ b(byte_copy_entry); + + __ bind(short_copy); + __ srwi(R5_count, R5_count, LogBytesPerShort); + __ b(short_copy_entry); + + __ bind(int_copy); + __ srwi(R5_count, R5_count, LogBytesPerInt); + __ b(int_copy_entry); + + __ bind(long_copy); + __ srwi(R5_count, R5_count, LogBytesPerLong); + __ b(long_copy_entry); + + return start; + } + + + // Perform range checks on the proposed arraycopy. + // Kills the two temps, but nothing else. + // Also, clean the sign bits of src_pos and dst_pos. + void arraycopy_range_checks(Register src, // source array oop + Register src_pos, // source position + Register dst, // destination array oop + Register dst_pos, // destination position + Register length, // length of copy + Register temp1, Register temp2, + Label& L_failed) { + BLOCK_COMMENT("arraycopy_range_checks:"); + + const Register array_length = temp1; // scratch + const Register end_pos = temp2; // scratch + + // if (src_pos + length > arrayOop(src)->length() ) FAIL; + __ lwa(array_length, arrayOopDesc::length_offset_in_bytes(), src); + __ add(end_pos, src_pos, length); // src_pos + length + __ cmpd(CCR0, end_pos, array_length); + __ bgt(CCR0, L_failed); + + // if (dst_pos + length > arrayOop(dst)->length() ) FAIL; + __ lwa(array_length, arrayOopDesc::length_offset_in_bytes(), dst); + __ add(end_pos, dst_pos, length); // src_pos + length + __ cmpd(CCR0, end_pos, array_length); + __ bgt(CCR0, L_failed); + + BLOCK_COMMENT("arraycopy_range_checks done"); + } + + + // + // Generate generic array copy stubs + // + // Input: + // R3 - src oop + // R4 - src_pos + // R5 - dst oop + // R6 - dst_pos + // R7 - element count + // + // Output: + // R3 == 0 - success + // R3 == -1 - need to call System.arraycopy + // + address generate_generic_copy(const char *name, + address entry_jbyte_arraycopy, + address entry_jshort_arraycopy, + address entry_jint_arraycopy, + address entry_oop_arraycopy, + address entry_disjoint_oop_arraycopy, + address entry_jlong_arraycopy, + address entry_checkcast_arraycopy) { + Label L_failed, L_objArray; + + // Input registers + const Register src = R3_ARG1; // source array oop + const Register src_pos = R4_ARG2; // source position + const Register dst = R5_ARG3; // destination array oop + const Register dst_pos = R6_ARG4; // destination position + const Register length = R7_ARG5; // elements count + + // registers used as temp + const Register src_klass = R8_ARG6; // source array klass + const Register dst_klass = R9_ARG7; // destination array klass + const Register lh = R10_ARG8; // layout handler + const Register temp = R2; + + //__ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", name); + address start = __ function_entry(); + + // Bump this on entry, not on exit: + //inc_counter_np(SharedRuntime::_generic_array_copy_ctr, lh, temp); + + // In principle, the int arguments could be dirty. + + //----------------------------------------------------------------------- + // Assembler stubs will be used for this call to arraycopy + // if the following conditions are met: + // + // (1) src and dst must not be null. + // (2) src_pos must not be negative. + // (3) dst_pos must not be negative. + // (4) length must not be negative. + // (5) src klass and dst klass should be the same and not NULL. + // (6) src and dst should be arrays. + // (7) src_pos + length must not exceed length of src. + // (8) dst_pos + length must not exceed length of dst. + BLOCK_COMMENT("arraycopy initial argument checks"); + + __ cmpdi(CCR1, src, 0); // if (src == NULL) return -1; + __ extsw_(src_pos, src_pos); // if (src_pos < 0) return -1; + __ cmpdi(CCR5, dst, 0); // if (dst == NULL) return -1; + __ cror(CCR1, Assembler::equal, CCR0, Assembler::less); + __ extsw_(dst_pos, dst_pos); // if (src_pos < 0) return -1; + __ cror(CCR5, Assembler::equal, CCR0, Assembler::less); + __ extsw_(length, length); // if (length < 0) return -1; + __ cror(CCR1, Assembler::equal, CCR5, Assembler::equal); + __ cror(CCR1, Assembler::equal, CCR0, Assembler::less); + __ beq(CCR1, L_failed); + + BLOCK_COMMENT("arraycopy argument klass checks"); + __ load_klass(src_klass, src); + __ load_klass(dst_klass, dst); + + // Load layout helper + // + // |array_tag| | header_size | element_type | |log2_element_size| + // 32 30 24 16 8 2 0 + // + // array_tag: typeArray = 0x3, objArray = 0x2, non-array = 0x0 + // + + int lh_offset = in_bytes(Klass::layout_helper_offset()); + + // Load 32-bits signed value. Use br() instruction with it to check icc. + __ lwz(lh, lh_offset, src_klass); + + // Handle objArrays completely differently... + jint objArray_lh = Klass::array_layout_helper(T_OBJECT); + __ load_const_optimized(temp, objArray_lh, R0); + __ cmpw(CCR0, lh, temp); + __ beq(CCR0, L_objArray); + + __ cmpd(CCR5, src_klass, dst_klass); // if (src->klass() != dst->klass()) return -1; + __ cmpwi(CCR6, lh, Klass::_lh_neutral_value); // if (!src->is_Array()) return -1; + + __ crnand(CCR5, Assembler::equal, CCR6, Assembler::less); + __ beq(CCR5, L_failed); + + // At this point, it is known to be a typeArray (array_tag 0x3). +#ifdef ASSERT + { Label L; + jint lh_prim_tag_in_place = (Klass::_lh_array_tag_type_value << Klass::_lh_array_tag_shift); + __ load_const_optimized(temp, lh_prim_tag_in_place, R0); + __ cmpw(CCR0, lh, temp); + __ bge(CCR0, L); + __ stop("must be a primitive array"); + __ bind(L); + } +#endif + + arraycopy_range_checks(src, src_pos, dst, dst_pos, length, + temp, dst_klass, L_failed); + + // TypeArrayKlass + // + // src_addr = (src + array_header_in_bytes()) + (src_pos << log2elemsize); + // dst_addr = (dst + array_header_in_bytes()) + (dst_pos << log2elemsize); + // + + const Register offset = dst_klass; // array offset + const Register elsize = src_klass; // log2 element size + + __ rldicl(offset, lh, 64 - Klass::_lh_header_size_shift, 64 - exact_log2(Klass::_lh_header_size_mask + 1)); + __ andi(elsize, lh, Klass::_lh_log2_element_size_mask); + __ add(src, offset, src); // src array offset + __ add(dst, offset, dst); // dst array offset + + // Next registers should be set before the jump to corresponding stub. + const Register from = R3_ARG1; // source array address + const Register to = R4_ARG2; // destination array address + const Register count = R5_ARG3; // elements count + + // 'from', 'to', 'count' registers should be set in this order + // since they are the same as 'src', 'src_pos', 'dst'. + + BLOCK_COMMENT("scale indexes to element size"); + __ sld(src_pos, src_pos, elsize); + __ sld(dst_pos, dst_pos, elsize); + __ add(from, src_pos, src); // src_addr + __ add(to, dst_pos, dst); // dst_addr + __ mr(count, length); // length + + BLOCK_COMMENT("choose copy loop based on element size"); + // Using conditional branches with range 32kB. + const int bo = Assembler::bcondCRbiIs1, bi = Assembler::bi0(CCR0, Assembler::equal); + __ cmpwi(CCR0, elsize, 0); + __ bc(bo, bi, entry_jbyte_arraycopy); + __ cmpwi(CCR0, elsize, LogBytesPerShort); + __ bc(bo, bi, entry_jshort_arraycopy); + __ cmpwi(CCR0, elsize, LogBytesPerInt); + __ bc(bo, bi, entry_jint_arraycopy); +#ifdef ASSERT + { Label L; + __ cmpwi(CCR0, elsize, LogBytesPerLong); + __ beq(CCR0, L); + __ stop("must be long copy, but elsize is wrong"); + __ bind(L); + } +#endif + __ b(entry_jlong_arraycopy); + + // ObjArrayKlass + __ bind(L_objArray); + // live at this point: src_klass, dst_klass, src[_pos], dst[_pos], length + + Label L_disjoint_plain_copy, L_checkcast_copy; + // test array classes for subtyping + __ cmpd(CCR0, src_klass, dst_klass); // usual case is exact equality + __ bne(CCR0, L_checkcast_copy); + + // Identically typed arrays can be copied without element-wise checks. + arraycopy_range_checks(src, src_pos, dst, dst_pos, length, + temp, lh, L_failed); + + __ addi(src, src, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); //src offset + __ addi(dst, dst, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); //dst offset + __ sldi(src_pos, src_pos, LogBytesPerHeapOop); + __ sldi(dst_pos, dst_pos, LogBytesPerHeapOop); + __ add(from, src_pos, src); // src_addr + __ add(to, dst_pos, dst); // dst_addr + __ mr(count, length); // length + __ b(entry_oop_arraycopy); + + __ bind(L_checkcast_copy); + // live at this point: src_klass, dst_klass + { + // Before looking at dst.length, make sure dst is also an objArray. + __ lwz(temp, lh_offset, dst_klass); + __ cmpw(CCR0, lh, temp); + __ bne(CCR0, L_failed); + + // It is safe to examine both src.length and dst.length. + arraycopy_range_checks(src, src_pos, dst, dst_pos, length, + temp, lh, L_failed); + + // Marshal the base address arguments now, freeing registers. + __ addi(src, src, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); //src offset + __ addi(dst, dst, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); //dst offset + __ sldi(src_pos, src_pos, LogBytesPerHeapOop); + __ sldi(dst_pos, dst_pos, LogBytesPerHeapOop); + __ add(from, src_pos, src); // src_addr + __ add(to, dst_pos, dst); // dst_addr + __ mr(count, length); // length + + Register sco_temp = R6_ARG4; // This register is free now. + assert_different_registers(from, to, count, sco_temp, + dst_klass, src_klass); + + // Generate the type check. + int sco_offset = in_bytes(Klass::super_check_offset_offset()); + __ lwz(sco_temp, sco_offset, dst_klass); + generate_type_check(src_klass, sco_temp, dst_klass, + temp, L_disjoint_plain_copy); + + // Fetch destination element klass from the ObjArrayKlass header. + int ek_offset = in_bytes(ObjArrayKlass::element_klass_offset()); + + // The checkcast_copy loop needs two extra arguments: + __ ld(R7_ARG5, ek_offset, dst_klass); // dest elem klass + __ lwz(R6_ARG4, sco_offset, R7_ARG5); // sco of elem klass + __ b(entry_checkcast_arraycopy); + } + + __ bind(L_disjoint_plain_copy); + __ b(entry_disjoint_oop_arraycopy); + + __ bind(L_failed); + __ li(R3_RET, -1); // return -1 + __ blr(); + return start; + } + + void generate_arraycopy_stubs() { // Note: the disjoint stubs must be generated first, some of // the conjoint stubs use them. @@ -1998,6 +2426,24 @@ class StubGenerator: public StubCodeGenerator { StubRoutines::_arrayof_oop_arraycopy = generate_conjoint_oop_copy(true, "arrayof_oop_arraycopy", false); StubRoutines::_arrayof_oop_arraycopy_uninit = generate_conjoint_oop_copy(true, "arrayof_oop_arraycopy", true); + // special/generic versions + StubRoutines::_checkcast_arraycopy = generate_checkcast_copy("checkcast_arraycopy", false); + StubRoutines::_checkcast_arraycopy_uninit = generate_checkcast_copy("checkcast_arraycopy_uninit", true); + + StubRoutines::_unsafe_arraycopy = generate_unsafe_copy("unsafe_arraycopy", + STUB_ENTRY(jbyte_arraycopy), + STUB_ENTRY(jshort_arraycopy), + STUB_ENTRY(jint_arraycopy), + STUB_ENTRY(jlong_arraycopy)); + StubRoutines::_generic_arraycopy = generate_generic_copy("generic_arraycopy", + STUB_ENTRY(jbyte_arraycopy), + STUB_ENTRY(jshort_arraycopy), + STUB_ENTRY(jint_arraycopy), + STUB_ENTRY(oop_arraycopy), + STUB_ENTRY(oop_disjoint_arraycopy), + STUB_ENTRY(jlong_arraycopy), + STUB_ENTRY(checkcast_arraycopy)); + // fill routines StubRoutines::_jbyte_fill = generate_fill(T_BYTE, false, "jbyte_fill"); StubRoutines::_jshort_fill = generate_fill(T_SHORT, false, "jshort_fill"); @@ -2228,6 +2674,15 @@ class StubGenerator: public StubCodeGenerator { StubRoutines::_multiplyToLen = generate_multiplyToLen(); } #endif + + if (UseMontgomeryMultiplyIntrinsic) { + StubRoutines::_montgomeryMultiply + = CAST_FROM_FN_PTR(address, SharedRuntime::montgomery_multiply); + } + if (UseMontgomerySquareIntrinsic) { + StubRoutines::_montgomerySquare + = CAST_FROM_FN_PTR(address, SharedRuntime::montgomery_square); + } } public: diff --git a/hotspot/src/cpu/ppc/vm/stubRoutines_ppc_64.cpp b/hotspot/src/cpu/ppc/vm/stubRoutines_ppc_64.cpp index 65239dd317d..1f49fe1de26 100644 --- a/hotspot/src/cpu/ppc/vm/stubRoutines_ppc_64.cpp +++ b/hotspot/src/cpu/ppc/vm/stubRoutines_ppc_64.cpp @@ -34,7 +34,7 @@ // CRC32 Intrinsics. void StubRoutines::ppc64::generate_load_crc_table_addr(MacroAssembler* masm, Register table) { - __ load_const(table, StubRoutines::_crc_table_adr); + __ load_const_optimized(table, StubRoutines::_crc_table_adr, R0); } // CRC32 Intrinsics. diff --git a/hotspot/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp b/hotspot/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp index 1972e21f743..eb00b5742c1 100644 --- a/hotspot/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/templateInterpreterGenerator_ppc.cpp @@ -254,34 +254,33 @@ void TemplateInterpreterGenerator::generate_counter_incr(Label* overflow, Label* if (TieredCompilation) { const int increment = InvocationCounter::count_increment; - const int mask = ((1 << Tier0InvokeNotifyFreqLog) - 1) << InvocationCounter::count_shift; Label no_mdo; if (ProfileInterpreter) { - const Register Rmdo = Rscratch1; + const Register Rmdo = R3_counters; // If no method data exists, go to profile_continue. __ ld(Rmdo, in_bytes(Method::method_data_offset()), R19_method); __ cmpdi(CCR0, Rmdo, 0); __ beq(CCR0, no_mdo); - // Increment backedge counter in the MDO. - const int mdo_bc_offs = in_bytes(MethodData::backedge_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); - __ lwz(Rscratch2, mdo_bc_offs, Rmdo); + // Increment invocation counter in the MDO. + const int mdo_ic_offs = in_bytes(MethodData::invocation_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); + __ lwz(Rscratch2, mdo_ic_offs, Rmdo); + __ lwz(Rscratch1, in_bytes(MethodData::invoke_mask_offset()), Rmdo); __ addi(Rscratch2, Rscratch2, increment); - __ stw(Rscratch2, mdo_bc_offs, Rmdo); - __ load_const_optimized(Rscratch1, mask, R0); + __ stw(Rscratch2, mdo_ic_offs, Rmdo); __ and_(Rscratch1, Rscratch2, Rscratch1); __ bne(CCR0, done); __ b(*overflow); } // Increment counter in MethodCounters*. - const int mo_bc_offs = in_bytes(MethodCounters::backedge_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); + const int mo_ic_offs = in_bytes(MethodCounters::invocation_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); __ bind(no_mdo); __ get_method_counters(R19_method, R3_counters, done); - __ lwz(Rscratch2, mo_bc_offs, R3_counters); + __ lwz(Rscratch2, mo_ic_offs, R3_counters); + __ lwz(Rscratch1, in_bytes(MethodCounters::invoke_mask_offset()), R3_counters); __ addi(Rscratch2, Rscratch2, increment); - __ stw(Rscratch2, mo_bc_offs, R3_counters); - __ load_const_optimized(Rscratch1, mask, R0); + __ stw(Rscratch2, mo_ic_offs, R3_counters); __ and_(Rscratch1, Rscratch2, Rscratch1); __ beq(CCR0, *overflow); @@ -302,8 +301,7 @@ void TemplateInterpreterGenerator::generate_counter_incr(Label* overflow, Label* // Check if we must create a method data obj. if (ProfileInterpreter && profile_method != NULL) { const Register profile_limit = Rscratch1; - int pl_offs = __ load_const_optimized(profile_limit, &InvocationCounter::InterpreterProfileLimit, R0, true); - __ lwz(profile_limit, pl_offs, profile_limit); + __ lwz(profile_limit, in_bytes(MethodCounters::interpreter_profile_limit_offset()), R3_counters); // Test to see if we should create a method data oop. __ cmpw(CCR0, Rsum_ivc_bec, profile_limit); __ blt(CCR0, *profile_method_continue); @@ -313,9 +311,7 @@ void TemplateInterpreterGenerator::generate_counter_incr(Label* overflow, Label* // Finally check for counter overflow. if (overflow) { const Register invocation_limit = Rscratch1; - int il_offs = __ load_const_optimized(invocation_limit, &InvocationCounter::InterpreterInvocationLimit, R0, true); - __ lwz(invocation_limit, il_offs, invocation_limit); - assert(4 == sizeof(InvocationCounter::InterpreterInvocationLimit), "unexpected field size"); + __ lwz(invocation_limit, in_bytes(MethodCounters::interpreter_invocation_limit_offset()), R3_counters); __ cmpw(CCR0, Rsum_ivc_bec, invocation_limit); __ bge(CCR0, *overflow); } diff --git a/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.cpp b/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.cpp index 8cd02dca4b3..cf0af66de9b 100644 --- a/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.cpp @@ -139,9 +139,9 @@ void AbstractInterpreter::layout_activation(Method* method, intptr_t* locals_base = (caller->is_interpreted_frame()) ? caller->interpreter_frame_esp() + caller_actual_parameters : - caller->sp() + method->max_locals() - 1 + (frame::abi_minframe_size / Interpreter::stackElementSize) ; + caller->sp() + method->max_locals() - 1 + (frame::abi_minframe_size / Interpreter::stackElementSize); - intptr_t* monitor_base = caller->sp() - frame::ijava_state_size / Interpreter::stackElementSize ; + intptr_t* monitor_base = caller->sp() - frame::ijava_state_size / Interpreter::stackElementSize; intptr_t* monitor = monitor_base - (moncount * frame::interpreter_frame_monitor_size()); intptr_t* esp_base = monitor - 1; intptr_t* esp = esp_base - tempcount - popframe_extra_args; diff --git a/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.hpp b/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.hpp new file mode 100644 index 00000000000..285b6f2306a --- /dev/null +++ b/hotspot/src/cpu/ppc/vm/templateInterpreter_ppc.hpp @@ -0,0 +1,42 @@ +/* + * Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013, 2015 SAP AG. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + * + */ + +#ifndef CPU_PPC_VM_TEMPLATEINTERPRETER_PPC_HPP +#define CPU_PPC_VM_TEMPLATEINTERPRETER_PPC_HPP + + protected: + + // Size of interpreter code. Increase if too small. Interpreter will + // fail with a guarantee ("not enough space for interpreter generation"); + // if too small. + // Run with +PrintInterpreter to get the VM to print out the size. + // Max size with JVMTI + const static int InterpreterCodeSize = 230*K; + + public: + // Support abs and sqrt like in compiler. + // For others we can use a normal (native) entry. + static bool math_entry_available(AbstractInterpreter::MethodKind kind); +#endif // CPU_PPC_VM_TEMPLATEINTERPRETER_PPC_HPP diff --git a/hotspot/src/cpu/ppc/vm/templateTable_ppc_64.cpp b/hotspot/src/cpu/ppc/vm/templateTable_ppc_64.cpp index 0bb08012e21..ec92898e890 100644 --- a/hotspot/src/cpu/ppc/vm/templateTable_ppc_64.cpp +++ b/hotspot/src/cpu/ppc/vm/templateTable_ppc_64.cpp @@ -1624,12 +1624,13 @@ void TemplateTable::branch(bool is_jsr, bool is_wide) { // -------------------------------------------------------------------------- // Normal (non-jsr) branch handling + // Bump bytecode pointer by displacement (take the branch). + __ add(R14_bcp, Rdisp, R14_bcp); // Add to bc addr. + const bool increment_invocation_counter_for_backward_branches = UseCompiler && UseLoopCounter; if (increment_invocation_counter_for_backward_branches) { - //__ unimplemented("branch invocation counter"); - Label Lforward; - __ add(R14_bcp, Rdisp, R14_bcp); // Add to bc addr. + __ dispatch_prolog(vtos); // Check branch direction. __ cmpdi(CCR0, Rdisp, 0); @@ -1640,7 +1641,6 @@ void TemplateTable::branch(bool is_jsr, bool is_wide) { if (TieredCompilation) { Label Lno_mdo, Loverflow; const int increment = InvocationCounter::count_increment; - const int mask = ((1 << Tier0BackedgeNotifyFreqLog) - 1) << InvocationCounter::count_shift; if (ProfileInterpreter) { Register Rmdo = Rscratch1; @@ -1652,7 +1652,7 @@ void TemplateTable::branch(bool is_jsr, bool is_wide) { // Increment backedge counter in the MDO. const int mdo_bc_offs = in_bytes(MethodData::backedge_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); __ lwz(Rscratch2, mdo_bc_offs, Rmdo); - __ load_const_optimized(Rscratch3, mask, R0); + __ lwz(Rscratch3, in_bytes(MethodData::backedge_mask_offset()), Rmdo); __ addi(Rscratch2, Rscratch2, increment); __ stw(Rscratch2, mdo_bc_offs, Rmdo); __ and_(Rscratch3, Rscratch2, Rscratch3); @@ -1664,19 +1664,19 @@ void TemplateTable::branch(bool is_jsr, bool is_wide) { const int mo_bc_offs = in_bytes(MethodCounters::backedge_counter_offset()) + in_bytes(InvocationCounter::counter_offset()); __ bind(Lno_mdo); __ lwz(Rscratch2, mo_bc_offs, R4_counters); - __ load_const_optimized(Rscratch3, mask, R0); + __ lwz(Rscratch3, in_bytes(MethodCounters::backedge_mask_offset()), R4_counters); __ addi(Rscratch2, Rscratch2, increment); - __ stw(Rscratch2, mo_bc_offs, R19_method); + __ stw(Rscratch2, mo_bc_offs, R4_counters); __ and_(Rscratch3, Rscratch2, Rscratch3); __ bne(CCR0, Lforward); __ bind(Loverflow); // Notify point for loop, pass branch bytecode. - __ call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::frequency_counter_overflow), R14_bcp, true); + __ subf(R4_ARG2, Rdisp, R14_bcp); // Compute branch bytecode (previous bcp). + __ call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::frequency_counter_overflow), R4_ARG2, true); // Was an OSR adapter generated? - // O0 = osr nmethod __ cmpdi(CCR0, R3_RET, 0); __ beq(CCR0, Lforward); @@ -1712,27 +1712,23 @@ void TemplateTable::branch(bool is_jsr, bool is_wide) { __ increment_backedge_counter(R4_counters, invoke_ctr, Rscratch2, Rscratch3); if (ProfileInterpreter) { - __ test_invocation_counter_for_mdp(invoke_ctr, Rscratch2, Lforward); + __ test_invocation_counter_for_mdp(invoke_ctr, R4_counters, Rscratch2, Lforward); if (UseOnStackReplacement) { - __ test_backedge_count_for_osr(bumped_count, R14_bcp, Rscratch2); + __ test_backedge_count_for_osr(bumped_count, R4_counters, R14_bcp, Rdisp, Rscratch2); } } else { if (UseOnStackReplacement) { - __ test_backedge_count_for_osr(invoke_ctr, R14_bcp, Rscratch2); + __ test_backedge_count_for_osr(invoke_ctr, R4_counters, R14_bcp, Rdisp, Rscratch2); } } } __ bind(Lforward); + __ dispatch_epilog(vtos); } else { - // Bump bytecode pointer by displacement (take the branch). - __ add(R14_bcp, Rdisp, R14_bcp); // Add to bc addr. + __ dispatch_next(vtos); } - // Continue with bytecode @ target. - // %%%%% Like Intel, could speed things up by moving bytecode fetch to code above, - // %%%%% and changing dispatch_next to dispatch_only. - __ dispatch_next(vtos); } // Helper function for if_cmp* methods below. diff --git a/hotspot/src/cpu/ppc/vm/vm_version_ppc.cpp b/hotspot/src/cpu/ppc/vm/vm_version_ppc.cpp index 443c26aba54..fec10b9268e 100644 --- a/hotspot/src/cpu/ppc/vm/vm_version_ppc.cpp +++ b/hotspot/src/cpu/ppc/vm/vm_version_ppc.cpp @@ -37,9 +37,6 @@ # include -int VM_Version::_features = VM_Version::unknown_m; -int VM_Version::_measured_cache_line_size = 32; // pessimistic init value -const char* VM_Version::_features_str = ""; bool VM_Version::_is_determine_features_test_running = false; @@ -56,7 +53,7 @@ void VM_Version::initialize() { // If PowerArchitecturePPC64 hasn't been specified explicitly determine from features. if (FLAG_IS_DEFAULT(PowerArchitecturePPC64)) { - if (VM_Version::has_lqarx()) { + if (VM_Version::has_tcheck() && VM_Version::has_lqarx()) { FLAG_SET_ERGO(uintx, PowerArchitecturePPC64, 8); } else if (VM_Version::has_popcntw()) { FLAG_SET_ERGO(uintx, PowerArchitecturePPC64, 7); @@ -68,10 +65,19 @@ void VM_Version::initialize() { FLAG_SET_ERGO(uintx, PowerArchitecturePPC64, 0); } } - guarantee(PowerArchitecturePPC64 == 0 || PowerArchitecturePPC64 == 5 || - PowerArchitecturePPC64 == 6 || PowerArchitecturePPC64 == 7 || - PowerArchitecturePPC64 == 8, - "PowerArchitecturePPC64 should be 0, 5, 6, 7, or 8"); + + bool PowerArchitecturePPC64_ok = false; + switch (PowerArchitecturePPC64) { + case 8: if (!VM_Version::has_tcheck() ) break; + if (!VM_Version::has_lqarx() ) break; + case 7: if (!VM_Version::has_popcntw()) break; + case 6: if (!VM_Version::has_cmpb() ) break; + case 5: if (!VM_Version::has_popcntb()) break; + case 0: PowerArchitecturePPC64_ok = true; break; + default: break; + } + guarantee(PowerArchitecturePPC64_ok, "PowerArchitecturePPC64 cannot be set to " + UINTX_FORMAT " on this machine", PowerArchitecturePPC64); // Power 8: Configure Data Stream Control Register. if (PowerArchitecturePPC64 >= 8) { @@ -122,7 +128,7 @@ void VM_Version::initialize() { (has_tcheck() ? " tcheck" : "") // Make sure number of %s matches num_features! ); - _features_str = os::strdup(buf); + _features_string = os::strdup(buf); if (Verbose) { print_features(); } @@ -132,9 +138,15 @@ void VM_Version::initialize() { // and 'atomic long memory ops' (see Unsafe_GetLongVolatile). _supports_cx8 = true; + // Used by C1. + _supports_atomic_getset4 = true; + _supports_atomic_getadd4 = true; + _supports_atomic_getset8 = true; + _supports_atomic_getadd8 = true; + UseSSE = 0; // Only on x86 and x64 - intx cache_line_size = _measured_cache_line_size; + intx cache_line_size = L1_data_cache_line_size(); if (FLAG_IS_DEFAULT(AllocatePrefetchStyle)) AllocatePrefetchStyle = 1; @@ -184,6 +196,11 @@ void VM_Version::initialize() { FLAG_SET_DEFAULT(UseAESIntrinsics, false); } + if (UseAESCTRIntrinsics) { + warning("AES/CTR intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + if (UseGHASHIntrinsics) { warning("GHASH intrinsics are not available on this CPU"); FLAG_SET_DEFAULT(UseGHASHIntrinsics, false); @@ -208,6 +225,18 @@ void VM_Version::initialize() { if (FLAG_IS_DEFAULT(UseMultiplyToLenIntrinsic)) { UseMultiplyToLenIntrinsic = true; } + if (FLAG_IS_DEFAULT(UseMontgomeryMultiplyIntrinsic)) { + UseMontgomeryMultiplyIntrinsic = true; + } + if (FLAG_IS_DEFAULT(UseMontgomerySquareIntrinsic)) { + UseMontgomerySquareIntrinsic = true; + } + + if (UseVectorizedMismatchIntrinsic) { + warning("UseVectorizedMismatchIntrinsic specified, but not available on this CPU."); + FLAG_SET_DEFAULT(UseVectorizedMismatchIntrinsic, false); + } + // Adjust RTM (Restricted Transactional Memory) flags. if (UseRTMLocking) { @@ -276,11 +305,9 @@ void VM_Version::initialize() { } } - // This machine does not allow unaligned memory accesses - if (UseUnalignedAccesses) { - if (!FLAG_IS_DEFAULT(UseUnalignedAccesses)) - warning("Unaligned memory access is not available on this CPU"); - FLAG_SET_DEFAULT(UseUnalignedAccesses, false); + // This machine allows unaligned memory accesses + if (FLAG_IS_DEFAULT(UseUnalignedAccesses)) { + FLAG_SET_DEFAULT(UseUnalignedAccesses, true); } } @@ -306,7 +333,7 @@ bool VM_Version::use_biased_locking() { } void VM_Version::print_features() { - tty->print_cr("Version: %s cache_line_size = %d", cpu_features(), (int) get_cache_line_size()); + tty->print_cr("Version: %s L1_data_cache_line_size=%d", features_string(), L1_data_cache_line_size()); } #ifdef COMPILER2 @@ -607,7 +634,7 @@ void VM_Version::determine_features() { int count = 0; // count zeroed bytes for (int i = 0; i < BUFFER_SIZE; i++) if (test_area[i] == 0) count++; guarantee(is_power_of_2(count), "cache line size needs to be a power of 2"); - _measured_cache_line_size = count; + _L1_data_cache_line_size = count; // Execute code. Illegal instructions will be replaced by 0 in the signal handler. VM_Version::_is_determine_features_test_running = true; @@ -705,7 +732,7 @@ void VM_Version::config_dscr() { } } -static int saved_features = 0; +static uint64_t saved_features = 0; void VM_Version::allow_all() { saved_features = _features; diff --git a/hotspot/src/cpu/ppc/vm/vm_version_ppc.hpp b/hotspot/src/cpu/ppc/vm/vm_version_ppc.hpp index 6fc76e4cd41..63b9ba09793 100644 --- a/hotspot/src/cpu/ppc/vm/vm_version_ppc.hpp +++ b/hotspot/src/cpu/ppc/vm/vm_version_ppc.hpp @@ -62,11 +62,9 @@ protected: vcipher_m = (1 << vcipher), vpmsumb_m = (1 << vpmsumb), tcheck_m = (1 << tcheck ), - all_features_m = -1 + all_features_m = (unsigned long)-1 }; - static int _features; - static int _measured_cache_line_size; - static const char* _features_str; + static bool _is_determine_features_test_running; static void print_features(); @@ -97,10 +95,6 @@ public: static bool has_vpmsumb() { return (_features & vpmsumb_m) != 0; } static bool has_tcheck() { return (_features & tcheck_m) != 0; } - static const char* cpu_features() { return _features_str; } - - static int get_cache_line_size() { return _measured_cache_line_size; } - // Assembler testing static void allow_all(); static void revert(); diff --git a/hotspot/src/cpu/ppc/vm/vtableStubs_ppc_64.cpp b/hotspot/src/cpu/ppc/vm/vtableStubs_ppc_64.cpp index 0165fb22e34..d19c211300f 100644 --- a/hotspot/src/cpu/ppc/vm/vtableStubs_ppc_64.cpp +++ b/hotspot/src/cpu/ppc/vm/vtableStubs_ppc_64.cpp @@ -76,7 +76,8 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) { // We might implicit NULL fault here. address npe_addr = __ pc(); // npe = null pointer exception - __ load_klass_with_trap_null_check(rcvr_klass, R3); + __ null_check(R3, oopDesc::klass_offset_in_bytes(), /*implicit only*/NULL); + __ load_klass(rcvr_klass, R3); // Set method (in case of interpreted method), and destination address. int entry_offset = InstanceKlass::vtable_start_offset() + vtable_index*vtableEntry::size(); @@ -111,8 +112,8 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) { // If the vtable entry is null, the method is abstract. address ame_addr = __ pc(); // ame = abstract method error - - __ load_with_trap_null_check(R12_scratch2, in_bytes(Method::from_compiled_offset()), R19_method); + __ null_check(R19_method, in_bytes(Method::from_compiled_offset()), /*implicit only*/NULL); + __ ld(R12_scratch2, in_bytes(Method::from_compiled_offset()), R19_method); __ mtctr(R12_scratch2); __ bctr(); masm->flush(); @@ -158,7 +159,8 @@ VtableStub* VtableStubs::create_itable_stub(int vtable_index) { // We might implicit NULL fault here. address npe_addr = __ pc(); // npe = null pointer exception - __ load_klass_with_trap_null_check(rcvr_klass, R3_ARG1); + __ null_check(R3_ARG1, oopDesc::klass_offset_in_bytes(), /*implicit only*/NULL); + __ load_klass(rcvr_klass, R3_ARG1); BLOCK_COMMENT("Load start of itable entries into itable_entry."); __ lwz(vtable_len, InstanceKlass::vtable_length_offset() * wordSize, rcvr_klass); @@ -217,15 +219,7 @@ VtableStub* VtableStubs::create_itable_stub(int vtable_index) { address ame_addr = __ pc(); // ame = abstract method error // Must do an explicit check if implicit checks are disabled. - assert(!MacroAssembler::needs_explicit_null_check(in_bytes(Method::from_compiled_offset())), "sanity"); - if (!ImplicitNullChecks || !os::zero_page_read_protected()) { - if (TrapBasedNullChecks) { - __ trap_null_check(R19_method); - } else { - __ cmpdi(CCR0, R19_method, 0); - __ beq(CCR0, throw_icce); - } - } + __ null_check(R19_method, in_bytes(Method::from_compiled_offset()), &throw_icce); __ ld(R12_scratch2, in_bytes(Method::from_compiled_offset()), R19_method); __ mtctr(R12_scratch2); __ bctr(); diff --git a/hotspot/src/cpu/sparc/vm/assembler_sparc.hpp b/hotspot/src/cpu/sparc/vm/assembler_sparc.hpp index e2ef96c727c..ec1a2423aa0 100644 --- a/hotspot/src/cpu/sparc/vm/assembler_sparc.hpp +++ b/hotspot/src/cpu/sparc/vm/assembler_sparc.hpp @@ -677,11 +677,8 @@ class Assembler : public AbstractAssembler { protected: // Insert a nop if the previous is cbcond - void insert_nop_after_cbcond() { - if (UseCBCond && cbcond_before()) { - nop(); - } - } + inline void insert_nop_after_cbcond(); + // Delay slot helpers // cti is called when emitting control-transfer instruction, // BEFORE doing the emitting. @@ -739,7 +736,7 @@ public: } inline void emit_int32(int); // shadows AbstractAssembler::emit_int32 - inline void emit_data(int x) { emit_int32(x); } + inline void emit_data(int x); inline void emit_data(int, RelocationHolder const&); inline void emit_data(int, relocInfo::relocType rtype); // helper for above fcns @@ -754,31 +751,31 @@ public: inline void add(Register s1, Register s2, Register d ); inline void add(Register s1, int simm13a, Register d ); - void addcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void addcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void addc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 ) | rs1(s1) | rs2(s2) ); } - void addc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void addccc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void addccc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void addcc( Register s1, Register s2, Register d ); + inline void addcc( Register s1, int simm13a, Register d ); + inline void addc( Register s1, Register s2, Register d ); + inline void addc( Register s1, int simm13a, Register d ); + inline void addccc( Register s1, Register s2, Register d ); + inline void addccc( Register s1, int simm13a, Register d ); // 4-operand AES instructions - void aes_eround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround01_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_eround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround23_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_dround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround01_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_dround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround23_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_eround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround01_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_eround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround23_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_dround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround01_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_dround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround23_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_kexpand1( FloatRegister s1, FloatRegister s2, int imm5a, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | u_field(imm5a, 13, 9) | op5(aes_kexpand1_op5) | fs2(s2, FloatRegisterImpl::D) ); } + inline void aes_eround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_eround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_dround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_dround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_eround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_eround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_dround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_dround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ); + inline void aes_kexpand1( FloatRegister s1, FloatRegister s2, int imm5a, FloatRegister d ); // 3-operand AES instructions - void aes_kexpand0( FloatRegister s1, FloatRegister s2, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes3_op3) | fs1(s1, FloatRegisterImpl::D) | opf(aes_kexpand0_opf) | fs2(s2, FloatRegisterImpl::D) ); } - void aes_kexpand2( FloatRegister s1, FloatRegister s2, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes3_op3) | fs1(s1, FloatRegisterImpl::D) | opf(aes_kexpand2_opf) | fs2(s2, FloatRegisterImpl::D) ); } + inline void aes_kexpand0( FloatRegister s1, FloatRegister s2, FloatRegister d ); + inline void aes_kexpand2( FloatRegister s1, FloatRegister s2, FloatRegister d ); // pp 136 @@ -816,6 +813,8 @@ public: inline void call( address d, relocInfo::relocType rt = relocInfo::runtime_call_type ); inline void call( Label& L, relocInfo::relocType rt = relocInfo::runtime_call_type ); + inline void call( address d, RelocationHolder const& rspec ); + public: // pp 150 @@ -825,70 +824,70 @@ public: // at address s1 is swapped with the data in d. If the values are not equal, // the the contents of memory at s1 is loaded into d, without the swap. - void casa( Register s1, Register s2, Register d, int ia = -1 ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(casa_op3 ) | rs1(s1) | (ia == -1 ? immed(true) : imm_asi(ia)) | rs2(s2)); } - void casxa( Register s1, Register s2, Register d, int ia = -1 ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(casxa_op3) | rs1(s1) | (ia == -1 ? immed(true) : imm_asi(ia)) | rs2(s2)); } + inline void casa( Register s1, Register s2, Register d, int ia = -1 ); + inline void casxa( Register s1, Register s2, Register d, int ia = -1 ); // pp 152 - void udiv( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 ) | rs1(s1) | rs2(s2)); } - void udiv( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void sdiv( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 ) | rs1(s1) | rs2(s2)); } - void sdiv( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void udivcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 | cc_bit_op3) | rs1(s1) | rs2(s2)); } - void udivcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void sdivcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 | cc_bit_op3) | rs1(s1) | rs2(s2)); } - void sdivcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void udiv( Register s1, Register s2, Register d ); + inline void udiv( Register s1, int simm13a, Register d ); + inline void sdiv( Register s1, Register s2, Register d ); + inline void sdiv( Register s1, int simm13a, Register d ); + inline void udivcc( Register s1, Register s2, Register d ); + inline void udivcc( Register s1, int simm13a, Register d ); + inline void sdivcc( Register s1, Register s2, Register d ); + inline void sdivcc( Register s1, int simm13a, Register d ); // pp 155 - void done() { v9_only(); cti(); emit_int32( op(arith_op) | fcn(0) | op3(done_op3) ); } - void retry() { v9_only(); cti(); emit_int32( op(arith_op) | fcn(1) | op3(retry_op3) ); } + inline void done(); + inline void retry(); // pp 156 - void fadd( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x40 + w) | fs2(s2, w)); } - void fsub( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x44 + w) | fs2(s2, w)); } + inline void fadd( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ); + inline void fsub( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ); // pp 157 - void fcmp( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2) { emit_int32( op(arith_op) | cmpcc(cc) | op3(fpop2_op3) | fs1(s1, w) | opf(0x50 + w) | fs2(s2, w)); } - void fcmpe( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2) { emit_int32( op(arith_op) | cmpcc(cc) | op3(fpop2_op3) | fs1(s1, w) | opf(0x54 + w) | fs2(s2, w)); } + inline void fcmp( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2); + inline void fcmpe( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2); // pp 159 - void ftox( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(fpop1_op3) | opf(0x80 + w) | fs2(s, w)); } - void ftoi( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::S) | op3(fpop1_op3) | opf(0xd0 + w) | fs2(s, w)); } + inline void ftox( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); + inline void ftoi( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); // pp 160 - void ftof( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, dw) | op3(fpop1_op3) | opf(0xc0 + sw + dw*4) | fs2(s, sw)); } + inline void ftof( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s, FloatRegister d ); // pp 161 - void fxtof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x80 + w*4) | fs2(s, FloatRegisterImpl::D)); } - void fitof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0xc0 + w*4) | fs2(s, FloatRegisterImpl::S)); } + inline void fxtof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); + inline void fitof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); // pp 162 - void fmov( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x00 + w) | fs2(s, w)); } + inline void fmov( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); - void fneg( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x04 + w) | fs2(s, w)); } + inline void fneg( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); - void fabs( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x08 + w) | fs2(s, w)); } + inline void fabs( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); // pp 163 - void fmul( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x48 + w) | fs2(s2, w)); } - void fmul( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, dw) | op3(fpop1_op3) | fs1(s1, sw) | opf(0x60 + sw + dw*4) | fs2(s2, sw)); } - void fdiv( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x4c + w) | fs2(s2, w)); } + inline void fmul( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ); + inline void fmul( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s1, FloatRegister s2, FloatRegister d ); + inline void fdiv( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ); // FXORs/FXORd instructions - void fxor( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(flog3_op3) | fs1(s1, w) | opf(0x6E - w) | fs2(s2, w)); } + inline void fxor( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ); // pp 164 - void fsqrt( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x28 + w) | fs2(s, w)); } + inline void fsqrt( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ); // pp 165 @@ -897,17 +896,17 @@ public: // pp 167 - void flushw() { v9_only(); emit_int32( op(arith_op) | op3(flushw_op3) ); } + void flushw(); // pp 168 - void illtrap( int const22a) { if (const22a != 0) v9_only(); emit_int32( op(branch_op) | u_field(const22a, 21, 0) ); } + void illtrap( int const22a); // v8 unimp == illtrap(0) // pp 169 - void impdep1( int id1, int const19a ) { v9_only(); emit_int32( op(arith_op) | fcn(id1) | op3(impdep1_op3) | u_field(const19a, 18, 0)); } - void impdep2( int id1, int const19a ) { v9_only(); emit_int32( op(arith_op) | fcn(id1) | op3(impdep2_op3) | u_field(const19a, 18, 0)); } + void impdep1( int id1, int const19a ); + void impdep2( int id1, int const19a ); // pp 170 @@ -927,8 +926,8 @@ public: // 173 - void ldfa( FloatRegisterImpl::Width w, Register s1, Register s2, int ia, FloatRegister d ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(ldf_op3 | alt_bit_op3, w) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void ldfa( FloatRegisterImpl::Width w, Register s1, int simm13a, FloatRegister d ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(ldf_op3 | alt_bit_op3, w) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void ldfa( FloatRegisterImpl::Width w, Register s1, Register s2, int ia, FloatRegister d ); + inline void ldfa( FloatRegisterImpl::Width w, Register s1, int simm13a, FloatRegister d ); // pp 175, lduw is ld on v8 @@ -951,119 +950,119 @@ public: // pp 177 - void ldsba( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsb_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void ldsba( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsb_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void ldsha( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsh_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void ldsha( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsh_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void ldswa( Register s1, Register s2, int ia, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldsw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void ldswa( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldsw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void lduba( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldub_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void lduba( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldub_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void lduha( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduh_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void lduha( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduh_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void lduwa( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void lduwa( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void ldxa( Register s1, Register s2, int ia, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldx_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void ldxa( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldx_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void ldsba( Register s1, Register s2, int ia, Register d ); + inline void ldsba( Register s1, int simm13a, Register d ); + inline void ldsha( Register s1, Register s2, int ia, Register d ); + inline void ldsha( Register s1, int simm13a, Register d ); + inline void ldswa( Register s1, Register s2, int ia, Register d ); + inline void ldswa( Register s1, int simm13a, Register d ); + inline void lduba( Register s1, Register s2, int ia, Register d ); + inline void lduba( Register s1, int simm13a, Register d ); + inline void lduha( Register s1, Register s2, int ia, Register d ); + inline void lduha( Register s1, int simm13a, Register d ); + inline void lduwa( Register s1, Register s2, int ia, Register d ); + inline void lduwa( Register s1, int simm13a, Register d ); + inline void ldxa( Register s1, Register s2, int ia, Register d ); + inline void ldxa( Register s1, int simm13a, Register d ); // pp 181 - void and3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 ) | rs1(s1) | rs2(s2) ); } - void and3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void andcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void andcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void andn( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 ) | rs1(s1) | rs2(s2) ); } - void andn( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void andncc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void andncc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void or3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 ) | rs1(s1) | rs2(s2) ); } - void or3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void orcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void orcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void orn( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3) | rs1(s1) | rs2(s2) ); } - void orn( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void orncc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void orncc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void xor3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 ) | rs1(s1) | rs2(s2) ); } - void xor3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void xorcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void xorcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void xnor( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 ) | rs1(s1) | rs2(s2) ); } - void xnor( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void xnorcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void xnorcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void and3( Register s1, Register s2, Register d ); + inline void and3( Register s1, int simm13a, Register d ); + inline void andcc( Register s1, Register s2, Register d ); + inline void andcc( Register s1, int simm13a, Register d ); + inline void andn( Register s1, Register s2, Register d ); + inline void andn( Register s1, int simm13a, Register d ); + inline void andncc( Register s1, Register s2, Register d ); + inline void andncc( Register s1, int simm13a, Register d ); + inline void or3( Register s1, Register s2, Register d ); + inline void or3( Register s1, int simm13a, Register d ); + inline void orcc( Register s1, Register s2, Register d ); + inline void orcc( Register s1, int simm13a, Register d ); + inline void orn( Register s1, Register s2, Register d ); + inline void orn( Register s1, int simm13a, Register d ); + inline void orncc( Register s1, Register s2, Register d ); + inline void orncc( Register s1, int simm13a, Register d ); + inline void xor3( Register s1, Register s2, Register d ); + inline void xor3( Register s1, int simm13a, Register d ); + inline void xorcc( Register s1, Register s2, Register d ); + inline void xorcc( Register s1, int simm13a, Register d ); + inline void xnor( Register s1, Register s2, Register d ); + inline void xnor( Register s1, int simm13a, Register d ); + inline void xnorcc( Register s1, Register s2, Register d ); + inline void xnorcc( Register s1, int simm13a, Register d ); // pp 183 - void membar( Membar_mask_bits const7a ) { v9_only(); emit_int32( op(arith_op) | op3(membar_op3) | rs1(O7) | immed(true) | u_field( int(const7a), 6, 0)); } + inline void membar( Membar_mask_bits const7a ); // pp 185 - void fmov( FloatRegisterImpl::Width w, Condition c, bool floatCC, CC cca, FloatRegister s2, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop2_op3) | cond_mov(c) | opf_cc(cca, floatCC) | opf_low6(w) | fs2(s2, w)); } + inline void fmov( FloatRegisterImpl::Width w, Condition c, bool floatCC, CC cca, FloatRegister s2, FloatRegister d ); // pp 189 - void fmov( FloatRegisterImpl::Width w, RCondition c, Register s1, FloatRegister s2, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop2_op3) | rs1(s1) | rcond(c) | opf_low5(4 + w) | fs2(s2, w)); } + inline void fmov( FloatRegisterImpl::Width w, RCondition c, Register s1, FloatRegister s2, FloatRegister d ); // pp 191 - void movcc( Condition c, bool floatCC, CC cca, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movcc_op3) | mov_cc(cca, floatCC) | cond_mov(c) | rs2(s2) ); } - void movcc( Condition c, bool floatCC, CC cca, int simm11a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movcc_op3) | mov_cc(cca, floatCC) | cond_mov(c) | immed(true) | simm(simm11a, 11) ); } + inline void movcc( Condition c, bool floatCC, CC cca, Register s2, Register d ); + inline void movcc( Condition c, bool floatCC, CC cca, int simm11a, Register d ); // pp 195 - void movr( RCondition c, Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movr_op3) | rs1(s1) | rcond(c) | rs2(s2) ); } - void movr( RCondition c, Register s1, int simm10a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movr_op3) | rs1(s1) | rcond(c) | immed(true) | simm(simm10a, 10) ); } + inline void movr( RCondition c, Register s1, Register s2, Register d ); + inline void movr( RCondition c, Register s1, int simm10a, Register d ); // pp 196 - void mulx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(mulx_op3 ) | rs1(s1) | rs2(s2) ); } - void mulx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(mulx_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void sdivx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sdivx_op3) | rs1(s1) | rs2(s2) ); } - void sdivx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sdivx_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void udivx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(udivx_op3) | rs1(s1) | rs2(s2) ); } - void udivx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(udivx_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void mulx( Register s1, Register s2, Register d ); + inline void mulx( Register s1, int simm13a, Register d ); + inline void sdivx( Register s1, Register s2, Register d ); + inline void sdivx( Register s1, int simm13a, Register d ); + inline void udivx( Register s1, Register s2, Register d ); + inline void udivx( Register s1, int simm13a, Register d ); // pp 197 - void umul( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 ) | rs1(s1) | rs2(s2) ); } - void umul( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void smul( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 ) | rs1(s1) | rs2(s2) ); } - void smul( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void umulcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void umulcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void smulcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void smulcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void umul( Register s1, Register s2, Register d ); + inline void umul( Register s1, int simm13a, Register d ); + inline void smul( Register s1, Register s2, Register d ); + inline void smul( Register s1, int simm13a, Register d ); + inline void umulcc( Register s1, Register s2, Register d ); + inline void umulcc( Register s1, int simm13a, Register d ); + inline void smulcc( Register s1, Register s2, Register d ); + inline void smulcc( Register s1, int simm13a, Register d ); // pp 201 - void nop() { emit_int32( op(branch_op) | op2(sethi_op2) ); } + inline void nop(); - void sw_count() { emit_int32( op(branch_op) | op2(sethi_op2) | 0x3f0 ); } + inline void sw_count(); // pp 202 - void popc( Register s, Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(popc_op3) | rs2(s)); } - void popc( int simm13a, Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(popc_op3) | immed(true) | simm(simm13a, 13)); } + inline void popc( Register s, Register d); + inline void popc( int simm13a, Register d); // pp 203 - void prefetch( Register s1, Register s2, PrefetchFcn f) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3) | rs1(s1) | rs2(s2) ); } - void prefetch( Register s1, int simm13a, PrefetchFcn f) { v9_only(); emit_data( op(ldst_op) | fcn(f) | op3(prefetch_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } + inline void prefetch( Register s1, Register s2, PrefetchFcn f); + inline void prefetch( Register s1, int simm13a, PrefetchFcn f); - void prefetcha( Register s1, Register s2, int ia, PrefetchFcn f ) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void prefetcha( Register s1, int simm13a, PrefetchFcn f ) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void prefetcha( Register s1, Register s2, int ia, PrefetchFcn f ); + inline void prefetcha( Register s1, int simm13a, PrefetchFcn f ); // pp 208 // not implementing read privileged register - inline void rdy( Register d) { v9_dep(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(0, 18, 14)); } - inline void rdccr( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(2, 18, 14)); } - inline void rdasi( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(3, 18, 14)); } - inline void rdtick( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(4, 18, 14)); } // Spoon! - inline void rdpc( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(5, 18, 14)); } - inline void rdfprs( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(6, 18, 14)); } + inline void rdy( Register d); + inline void rdccr( Register d); + inline void rdasi( Register d); + inline void rdtick( Register d); + inline void rdpc( Register d); + inline void rdfprs( Register d); // pp 213 @@ -1072,47 +1071,43 @@ public: // pp 214 - void save( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(save_op3) | rs1(s1) | rs2(s2) ); } - void save( Register s1, int simm13a, Register d ) { - // make sure frame is at least large enough for the register save area - assert(-simm13a >= 16 * wordSize, "frame too small"); - emit_int32( op(arith_op) | rd(d) | op3(save_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); - } + inline void save( Register s1, Register s2, Register d ); + inline void save( Register s1, int simm13a, Register d ); - void restore( Register s1 = G0, Register s2 = G0, Register d = G0 ) { emit_int32( op(arith_op) | rd(d) | op3(restore_op3) | rs1(s1) | rs2(s2) ); } - void restore( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(restore_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void restore( Register s1 = G0, Register s2 = G0, Register d = G0 ); + inline void restore( Register s1, int simm13a, Register d ); // pp 216 - void saved() { v9_only(); emit_int32( op(arith_op) | fcn(0) | op3(saved_op3)); } - void restored() { v9_only(); emit_int32( op(arith_op) | fcn(1) | op3(saved_op3)); } + inline void saved(); + inline void restored(); // pp 217 inline void sethi( int imm22a, Register d, RelocationHolder const& rspec = RelocationHolder() ); // pp 218 - void sll( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(0) | rs2(s2) ); } - void sll( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } - void srl( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(0) | rs2(s2) ); } - void srl( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } - void sra( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(0) | rs2(s2) ); } - void sra( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } + inline void sll( Register s1, Register s2, Register d ); + inline void sll( Register s1, int imm5a, Register d ); + inline void srl( Register s1, Register s2, Register d ); + inline void srl( Register s1, int imm5a, Register d ); + inline void sra( Register s1, Register s2, Register d ); + inline void sra( Register s1, int imm5a, Register d ); - void sllx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(1) | rs2(s2) ); } - void sllx( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } - void srlx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(1) | rs2(s2) ); } - void srlx( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } - void srax( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(1) | rs2(s2) ); } - void srax( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } + inline void sllx( Register s1, Register s2, Register d ); + inline void sllx( Register s1, int imm6a, Register d ); + inline void srlx( Register s1, Register s2, Register d ); + inline void srlx( Register s1, int imm6a, Register d ); + inline void srax( Register s1, Register s2, Register d ); + inline void srax( Register s1, int imm6a, Register d ); // pp 220 - void sir( int simm13a ) { emit_int32( op(arith_op) | fcn(15) | op3(sir_op3) | immed(true) | simm(simm13a, 13)); } + inline void sir( int simm13a ); // pp 221 - void stbar() { emit_int32( op(arith_op) | op3(membar_op3) | u_field(15, 18, 14)); } + inline void stbar(); // pp 222 @@ -1126,8 +1121,8 @@ public: // pp 224 - void stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, Register s2, int ia ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(stf_op3 | alt_bit_op3, w) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, int simm13a ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(stf_op3 | alt_bit_op3, w) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, Register s2, int ia ); + inline void stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, int simm13a ); // p 226 @@ -1144,28 +1139,28 @@ public: // pp 177 - void stba( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(stb_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stba( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(stb_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void stha( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(sth_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stha( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(sth_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void stwa( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(stw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stwa( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(stw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void stxa( Register d, Register s1, Register s2, int ia ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(stx_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stxa( Register d, Register s1, int simm13a ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(stx_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void stda( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(std_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void stda( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(std_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void stba( Register d, Register s1, Register s2, int ia ); + inline void stba( Register d, Register s1, int simm13a ); + inline void stha( Register d, Register s1, Register s2, int ia ); + inline void stha( Register d, Register s1, int simm13a ); + inline void stwa( Register d, Register s1, Register s2, int ia ); + inline void stwa( Register d, Register s1, int simm13a ); + inline void stxa( Register d, Register s1, Register s2, int ia ); + inline void stxa( Register d, Register s1, int simm13a ); + inline void stda( Register d, Register s1, Register s2, int ia ); + inline void stda( Register d, Register s1, int simm13a ); // pp 230 - void sub( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 ) | rs1(s1) | rs2(s2) ); } - void sub( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void sub( Register s1, Register s2, Register d ); + inline void sub( Register s1, int simm13a, Register d ); - void subcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 | cc_bit_op3 ) | rs1(s1) | rs2(s2) ); } - void subcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 | cc_bit_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void subc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 ) | rs1(s1) | rs2(s2) ); } - void subc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } - void subccc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } - void subccc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void subcc( Register s1, Register s2, Register d ); + inline void subcc( Register s1, int simm13a, Register d ); + inline void subc( Register s1, Register s2, Register d ); + inline void subc( Register s1, int simm13a, Register d ); + inline void subccc( Register s1, Register s2, Register d ); + inline void subccc( Register s1, int simm13a, Register d ); // pp 231 @@ -1174,86 +1169,80 @@ public: // pp 232 - void swapa( Register s1, Register s2, int ia, Register d ) { v9_dep(); emit_int32( op(ldst_op) | rd(d) | op3(swap_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } - void swapa( Register s1, int simm13a, Register d ) { v9_dep(); emit_int32( op(ldst_op) | rd(d) | op3(swap_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void swapa( Register s1, Register s2, int ia, Register d ); + inline void swapa( Register s1, int simm13a, Register d ); // pp 234, note op in book is wrong, see pp 268 - void taddcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(taddcc_op3 ) | rs1(s1) | rs2(s2) ); } - void taddcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(taddcc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void taddcc( Register s1, Register s2, Register d ); + inline void taddcc( Register s1, int simm13a, Register d ); // pp 235 - void tsubcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(tsubcc_op3 ) | rs1(s1) | rs2(s2) ); } - void tsubcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(tsubcc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void tsubcc( Register s1, Register s2, Register d ); + inline void tsubcc( Register s1, int simm13a, Register d ); // pp 237 - void trap( Condition c, CC cc, Register s1, Register s2 ) { emit_int32( op(arith_op) | cond(c) | op3(trap_op3) | rs1(s1) | trapcc(cc) | rs2(s2)); } - void trap( Condition c, CC cc, Register s1, int trapa ) { emit_int32( op(arith_op) | cond(c) | op3(trap_op3) | rs1(s1) | trapcc(cc) | immed(true) | u_field(trapa, 6, 0)); } + inline void trap( Condition c, CC cc, Register s1, Register s2 ); + inline void trap( Condition c, CC cc, Register s1, int trapa ); // simple uncond. trap - void trap( int trapa ) { trap( always, icc, G0, trapa ); } + inline void trap( int trapa ); // pp 239 omit write priv register for now - inline void wry( Register d) { v9_dep(); emit_int32( op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(0, 29, 25)); } - inline void wrccr(Register s) { v9_only(); emit_int32( op(arith_op) | rs1(s) | op3(wrreg_op3) | u_field(2, 29, 25)); } - inline void wrccr(Register s, int simm13a) { v9_only(); emit_int32( op(arith_op) | - rs1(s) | - op3(wrreg_op3) | - u_field(2, 29, 25) | - immed(true) | - simm(simm13a, 13)); } - inline void wrasi(Register d) { v9_only(); emit_int32( op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(3, 29, 25)); } + inline void wry( Register d); + inline void wrccr(Register s); + inline void wrccr(Register s, int simm13a); + inline void wrasi(Register d); // wrasi(d, imm) stores (d xor imm) to asi - inline void wrasi(Register d, int simm13a) { v9_only(); emit_int32( op(arith_op) | rs1(d) | op3(wrreg_op3) | - u_field(3, 29, 25) | immed(true) | simm(simm13a, 13)); } - inline void wrfprs( Register d) { v9_only(); emit_int32( op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(6, 29, 25)); } + inline void wrasi(Register d, int simm13a); + inline void wrfprs( Register d); // VIS1 instructions - void alignaddr( Register s1, Register s2, Register d ) { vis1_only(); emit_int32( op(arith_op) | rd(d) | op3(alignaddr_op3) | rs1(s1) | opf(alignaddr_opf) | rs2(s2)); } + inline void alignaddr( Register s1, Register s2, Register d ); - void faligndata( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(faligndata_op3) | fs1(s1, FloatRegisterImpl::D) | opf(faligndata_opf) | fs2(s2, FloatRegisterImpl::D)); } + inline void faligndata( FloatRegister s1, FloatRegister s2, FloatRegister d ); - void fzero( FloatRegisterImpl::Width w, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fzero_op3) | opf(0x62 - w)); } + inline void fzero( FloatRegisterImpl::Width w, FloatRegister d ); - void fsrc2( FloatRegisterImpl::Width w, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fsrc_op3) | opf(0x7A - w) | fs2(s2, w)); } + inline void fsrc2( FloatRegisterImpl::Width w, FloatRegister s2, FloatRegister d ); - void fnot1( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fnot_op3) | fs1(s1, w) | opf(0x6C - w)); } + inline void fnot1( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister d ); - void fpmerge( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(0x36) | fs1(s1, FloatRegisterImpl::S) | opf(0x4b) | fs2(s2, FloatRegisterImpl::S)); } + inline void fpmerge( FloatRegister s1, FloatRegister s2, FloatRegister d ); - void stpartialf( Register s1, Register s2, FloatRegister d, int ia = -1 ) { vis1_only(); emit_int32( op(ldst_op) | fd(d, FloatRegisterImpl::D) | op3(stpartialf_op3) | rs1(s1) | imm_asi(ia) | rs2(s2)); } + inline void stpartialf( Register s1, Register s2, FloatRegister d, int ia = -1 ); // VIS2 instructions - void edge8n( Register s1, Register s2, Register d ) { vis2_only(); emit_int32( op(arith_op) | rd(d) | op3(edge_op3) | rs1(s1) | opf(edge8n_opf) | rs2(s2)); } + inline void edge8n( Register s1, Register s2, Register d ); - void bmask( Register s1, Register s2, Register d ) { vis2_only(); emit_int32( op(arith_op) | rd(d) | op3(bmask_op3) | rs1(s1) | opf(bmask_opf) | rs2(s2)); } - void bshuffle( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis2_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(bshuffle_op3) | fs1(s1, FloatRegisterImpl::D) | opf(bshuffle_opf) | fs2(s2, FloatRegisterImpl::D)); } + inline void bmask( Register s1, Register s2, Register d ); + inline void bshuffle( FloatRegister s1, FloatRegister s2, FloatRegister d ); // VIS3 instructions - void movstosw( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mstosw_opf) | fs2(s, FloatRegisterImpl::S)); } - void movstouw( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mstouw_opf) | fs2(s, FloatRegisterImpl::S)); } - void movdtox( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mdtox_opf) | fs2(s, FloatRegisterImpl::D)); } + inline void movstosw( FloatRegister s, Register d ); + inline void movstouw( FloatRegister s, Register d ); + inline void movdtox( FloatRegister s, Register d ); - void movwtos( Register s, FloatRegister d ) { vis3_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::S) | op3(mftoi_op3) | opf(mwtos_opf) | rs2(s)); } - void movxtod( Register s, FloatRegister d ) { vis3_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(mftoi_op3) | opf(mxtod_opf) | rs2(s)); } + inline void movwtos( Register s, FloatRegister d ); + inline void movxtod( Register s, FloatRegister d ); - void xmulx(Register s1, Register s2, Register d) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(xmulx_op3) | rs1(s1) | opf(xmulx_opf) | rs2(s2)); } - void xmulxhi(Register s1, Register s2, Register d) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(xmulx_op3) | rs1(s1) | opf(xmulxhi_opf) | rs2(s2)); } + inline void xmulx(Register s1, Register s2, Register d); + inline void xmulxhi(Register s1, Register s2, Register d); // Crypto SHA instructions - void sha1() { sha1_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha1_opf)); } - void sha256() { sha256_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha256_opf)); } - void sha512() { sha512_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha512_opf)); } + inline void sha1(); + inline void sha256(); + inline void sha512(); // CRC32C instruction - void crc32c( FloatRegister s1, FloatRegister s2, FloatRegister d ) { crc32c_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(crc32c_op3) | fs1(s1, FloatRegisterImpl::D) | opf(crc32c_opf) | fs2(s2, FloatRegisterImpl::D)); } + inline void crc32c( FloatRegister s1, FloatRegister s2, FloatRegister d ); // Creation Assembler(CodeBuffer* code) : AbstractAssembler(code) { diff --git a/hotspot/src/cpu/sparc/vm/assembler_sparc.inline.hpp b/hotspot/src/cpu/sparc/vm/assembler_sparc.inline.hpp index 2bbf95e3b61..c18b07ec019 100644 --- a/hotspot/src/cpu/sparc/vm/assembler_sparc.inline.hpp +++ b/hotspot/src/cpu/sparc/vm/assembler_sparc.inline.hpp @@ -28,6 +28,12 @@ #include "asm/assembler.hpp" +inline void Assembler::insert_nop_after_cbcond() { + if (UseCBCond && cbcond_before()) { + nop(); + } +} + inline void Assembler::check_delay() { # ifdef CHECK_DELAY guarantee( delay_state != at_delay_slot, "must say delayed() when filling delay slot"); @@ -40,6 +46,10 @@ inline void Assembler::emit_int32(int x) { AbstractAssembler::emit_int32(x); } +inline void Assembler::emit_data(int x) { + emit_int32(x); +} + inline void Assembler::emit_data(int x, relocInfo::relocType rtype) { relocate(rtype); emit_int32(x); @@ -54,6 +64,29 @@ inline void Assembler::emit_data(int x, RelocationHolder const& rspec) { inline void Assembler::add(Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::add(Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::addcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::addcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(add_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::addc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::addc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::addccc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::addccc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(addc_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::aes_eround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround01_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_eround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround23_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_dround01( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround01_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_dround23( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround23_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_eround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround01_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_eround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_eround23_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_dround01_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround01_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_dround23_l( FloatRegister s1, FloatRegister s2, FloatRegister s3, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | fs3(s3, FloatRegisterImpl::D) | op5(aes_dround23_l_op5) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_kexpand1( FloatRegister s1, FloatRegister s2, int imm5a, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes4_op3) | fs1(s1, FloatRegisterImpl::D) | u_field(imm5a, 13, 9) | op5(aes_kexpand1_op5) | fs2(s2, FloatRegisterImpl::D) ); } + + +// 3-operand AES instructions + +inline void Assembler::aes_kexpand0( FloatRegister s1, FloatRegister s2, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes3_op3) | fs1(s1, FloatRegisterImpl::D) | opf(aes_kexpand0_opf) | fs2(s2, FloatRegisterImpl::D) ); } +inline void Assembler::aes_kexpand2( FloatRegister s1, FloatRegister s2, FloatRegister d ) { aes_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(aes3_op3) | fs1(s1, FloatRegisterImpl::D) | opf(aes_kexpand2_opf) | fs2(s2, FloatRegisterImpl::D) ); } + inline void Assembler::bpr( RCondition c, bool a, Predict p, Register s1, address d, relocInfo::relocType rt ) { v9_only(); insert_nop_after_cbcond(); cti(); emit_data( op(branch_op) | annul(a) | cond(c) | op2(bpr_op2) | wdisp16(intptr_t(d), intptr_t(pc())) | predict(p) | rs1(s1), rt); has_delay_slot(); } inline void Assembler::bpr( RCondition c, bool a, Predict p, Register s1, Label& L) { insert_nop_after_cbcond(); bpr( c, a, p, s1, target(L)); } @@ -76,9 +109,58 @@ inline void Assembler::cbcond(Condition c, CC cc, Register s1, int simm5, Label& inline void Assembler::call( address d, relocInfo::relocType rt ) { insert_nop_after_cbcond(); cti(); emit_data( op(call_op) | wdisp(intptr_t(d), intptr_t(pc()), 30), rt); has_delay_slot(); assert(rt != relocInfo::virtual_call_type, "must use virtual_call_Relocation::spec"); } inline void Assembler::call( Label& L, relocInfo::relocType rt ) { insert_nop_after_cbcond(); call( target(L), rt); } +inline void Assembler::call( address d, RelocationHolder const& rspec ) { insert_nop_after_cbcond(); cti(); emit_data( op(call_op) | wdisp(intptr_t(d), intptr_t(pc()), 30), rspec); has_delay_slot(); assert(rspec.type() != relocInfo::virtual_call_type, "must use virtual_call_Relocation::spec"); } + +inline void Assembler::casa( Register s1, Register s2, Register d, int ia ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(casa_op3 ) | rs1(s1) | (ia == -1 ? immed(true) : imm_asi(ia)) | rs2(s2)); } +inline void Assembler::casxa( Register s1, Register s2, Register d, int ia ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(casxa_op3) | rs1(s1) | (ia == -1 ? immed(true) : imm_asi(ia)) | rs2(s2)); } + +inline void Assembler::udiv( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 ) | rs1(s1) | rs2(s2)); } +inline void Assembler::udiv( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::sdiv( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 ) | rs1(s1) | rs2(s2)); } +inline void Assembler::sdiv( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::udivcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 | cc_bit_op3) | rs1(s1) | rs2(s2)); } +inline void Assembler::udivcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(udiv_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::sdivcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 | cc_bit_op3) | rs1(s1) | rs2(s2)); } +inline void Assembler::sdivcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sdiv_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::done() { v9_only(); cti(); emit_int32( op(arith_op) | fcn(0) | op3(done_op3) ); } +inline void Assembler::retry() { v9_only(); cti(); emit_int32( op(arith_op) | fcn(1) | op3(retry_op3) ); } + +inline void Assembler::fadd( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x40 + w) | fs2(s2, w)); } +inline void Assembler::fsub( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x44 + w) | fs2(s2, w)); } + +inline void Assembler::fcmp( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2) { emit_int32( op(arith_op) | cmpcc(cc) | op3(fpop2_op3) | fs1(s1, w) | opf(0x50 + w) | fs2(s2, w)); } +inline void Assembler::fcmpe( FloatRegisterImpl::Width w, CC cc, FloatRegister s1, FloatRegister s2) { emit_int32( op(arith_op) | cmpcc(cc) | op3(fpop2_op3) | fs1(s1, w) | opf(0x54 + w) | fs2(s2, w)); } + +inline void Assembler::ftox( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(fpop1_op3) | opf(0x80 + w) | fs2(s, w)); } +inline void Assembler::ftoi( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::S) | op3(fpop1_op3) | opf(0xd0 + w) | fs2(s, w)); } + +inline void Assembler::ftof( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, dw) | op3(fpop1_op3) | opf(0xc0 + sw + dw*4) | fs2(s, sw)); } + +inline void Assembler::fxtof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x80 + w*4) | fs2(s, FloatRegisterImpl::D)); } +inline void Assembler::fitof( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0xc0 + w*4) | fs2(s, FloatRegisterImpl::S)); } + +inline void Assembler::fmov( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x00 + w) | fs2(s, w)); } +inline void Assembler::fneg( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x04 + w) | fs2(s, w)); } +inline void Assembler::fabs( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x08 + w) | fs2(s, w)); } +inline void Assembler::fmul( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x48 + w) | fs2(s2, w)); } +inline void Assembler::fmul( FloatRegisterImpl::Width sw, FloatRegisterImpl::Width dw, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, dw) | op3(fpop1_op3) | fs1(s1, sw) | opf(0x60 + sw + dw*4) | fs2(s2, sw)); } +inline void Assembler::fdiv( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | fs1(s1, w) | opf(0x4c + w) | fs2(s2, w)); } + +inline void Assembler::fxor( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(flog3_op3) | fs1(s1, w) | opf(0x6E - w) | fs2(s2, w)); } + +inline void Assembler::fsqrt( FloatRegisterImpl::Width w, FloatRegister s, FloatRegister d ) { emit_int32( op(arith_op) | fd(d, w) | op3(fpop1_op3) | opf(0x28 + w) | fs2(s, w)); } + inline void Assembler::flush( Register s1, Register s2) { emit_int32( op(arith_op) | op3(flush_op3) | rs1(s1) | rs2(s2)); } inline void Assembler::flush( Register s1, int simm13a) { emit_data( op(arith_op) | op3(flush_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::flushw() { v9_only(); emit_int32( op(arith_op) | op3(flushw_op3) ); } + +inline void Assembler::illtrap( int const22a) { if (const22a != 0) v9_only(); emit_int32( op(branch_op) | u_field(const22a, 21, 0) ); } + +inline void Assembler::impdep1( int id1, int const19a ) { v9_only(); emit_int32( op(arith_op) | fcn(id1) | op3(impdep1_op3) | u_field(const19a, 18, 0)); } +inline void Assembler::impdep2( int id1, int const19a ) { v9_only(); emit_int32( op(arith_op) | fcn(id1) | op3(impdep2_op3) | u_field(const19a, 18, 0)); } + inline void Assembler::jmpl( Register s1, Register s2, Register d ) { insert_nop_after_cbcond(); cti(); emit_int32( op(arith_op) | rd(d) | op3(jmpl_op3) | rs1(s1) | rs2(s2)); has_delay_slot(); } inline void Assembler::jmpl( Register s1, int simm13a, Register d, RelocationHolder const& rspec ) { insert_nop_after_cbcond(); cti(); emit_data( op(arith_op) | rd(d) | op3(jmpl_op3) | rs1(s1) | immed(true) | simm(simm13a, 13), rspec); has_delay_slot(); } @@ -88,6 +170,9 @@ inline void Assembler::ldf(FloatRegisterImpl::Width w, Register s1, int simm13a, inline void Assembler::ldxfsr( Register s1, Register s2) { v9_only(); emit_int32( op(ldst_op) | rd(G1) | op3(ldfsr_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::ldxfsr( Register s1, int simm13a) { v9_only(); emit_data( op(ldst_op) | rd(G1) | op3(ldfsr_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::ldfa( FloatRegisterImpl::Width w, Register s1, Register s2, int ia, FloatRegister d ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(ldf_op3 | alt_bit_op3, w) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::ldfa( FloatRegisterImpl::Width w, Register s1, int simm13a, FloatRegister d ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(ldf_op3 | alt_bit_op3, w) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + inline void Assembler::ldsb( Register s1, Register s2, Register d) { emit_int32( op(ldst_op) | rd(d) | op3(ldsb_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::ldsb( Register s1, int simm13a, Register d) { emit_data( op(ldst_op) | rd(d) | op3(ldsb_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } @@ -107,11 +192,134 @@ inline void Assembler::ldx( Register s1, int simm13a, Register d) { v9_only(); inline void Assembler::ldd( Register s1, Register s2, Register d) { v9_dep(); assert(d->is_even(), "not even"); emit_int32( op(ldst_op) | rd(d) | op3(ldd_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::ldd( Register s1, int simm13a, Register d) { v9_dep(); assert(d->is_even(), "not even"); emit_data( op(ldst_op) | rd(d) | op3(ldd_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::ldsba( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsb_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::ldsba( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsb_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::ldsha( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsh_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::ldsha( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldsh_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::ldswa( Register s1, Register s2, int ia, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldsw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::ldswa( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldsw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::lduba( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldub_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::lduba( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(ldub_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::lduha( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduh_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::lduha( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduh_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::lduwa( Register s1, Register s2, int ia, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::lduwa( Register s1, int simm13a, Register d ) { emit_int32( op(ldst_op) | rd(d) | op3(lduw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::ldxa( Register s1, Register s2, int ia, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldx_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::ldxa( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(ldx_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::and3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::and3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::andcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::andcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(and_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::andn( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::andn( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::andncc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::andncc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(andn_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::or3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::or3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::orcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::orcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(or_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::orn( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::orn( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::orncc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::orncc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(orn_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::xor3( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::xor3( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::xorcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::xorcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xor_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::xnor( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::xnor( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::xnorcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::xnorcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(xnor_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::membar( Membar_mask_bits const7a ) { v9_only(); emit_int32( op(arith_op) | op3(membar_op3) | rs1(O7) | immed(true) | u_field( int(const7a), 6, 0)); } + +inline void Assembler::fmov( FloatRegisterImpl::Width w, Condition c, bool floatCC, CC cca, FloatRegister s2, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop2_op3) | cond_mov(c) | opf_cc(cca, floatCC) | opf_low6(w) | fs2(s2, w)); } + +inline void Assembler::fmov( FloatRegisterImpl::Width w, RCondition c, Register s1, FloatRegister s2, FloatRegister d ) { v9_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fpop2_op3) | rs1(s1) | rcond(c) | opf_low5(4 + w) | fs2(s2, w)); } + +inline void Assembler::movcc( Condition c, bool floatCC, CC cca, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movcc_op3) | mov_cc(cca, floatCC) | cond_mov(c) | rs2(s2) ); } +inline void Assembler::movcc( Condition c, bool floatCC, CC cca, int simm11a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movcc_op3) | mov_cc(cca, floatCC) | cond_mov(c) | immed(true) | simm(simm11a, 11) ); } + +inline void Assembler::movr( RCondition c, Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movr_op3) | rs1(s1) | rcond(c) | rs2(s2) ); } +inline void Assembler::movr( RCondition c, Register s1, int simm10a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(movr_op3) | rs1(s1) | rcond(c) | immed(true) | simm(simm10a, 10) ); } + +inline void Assembler::mulx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(mulx_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::mulx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(mulx_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::sdivx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sdivx_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::sdivx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sdivx_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::udivx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(udivx_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::udivx( Register s1, int simm13a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(udivx_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::umul( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::umul( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::smul( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::smul( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::umulcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::umulcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(umul_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::smulcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::smulcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(smul_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::nop() { emit_int32( op(branch_op) | op2(sethi_op2) ); } + +inline void Assembler::sw_count() { emit_int32( op(branch_op) | op2(sethi_op2) | 0x3f0 ); } + +inline void Assembler::popc( Register s, Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(popc_op3) | rs2(s)); } +inline void Assembler::popc( int simm13a, Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(popc_op3) | immed(true) | simm(simm13a, 13)); } + +inline void Assembler::prefetch( Register s1, Register s2, PrefetchFcn f) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::prefetch( Register s1, int simm13a, PrefetchFcn f) { v9_only(); emit_data( op(ldst_op) | fcn(f) | op3(prefetch_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } + +inline void Assembler::prefetcha( Register s1, Register s2, int ia, PrefetchFcn f ) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::prefetcha( Register s1, int simm13a, PrefetchFcn f ) { v9_only(); emit_int32( op(ldst_op) | fcn(f) | op3(prefetch_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::rdy( Register d) { v9_dep(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(0, 18, 14)); } +inline void Assembler::rdccr( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(2, 18, 14)); } +inline void Assembler::rdasi( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(3, 18, 14)); } +inline void Assembler::rdtick( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(4, 18, 14)); } // Spoon! +inline void Assembler::rdpc( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(5, 18, 14)); } +inline void Assembler::rdfprs( Register d) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(rdreg_op3) | u_field(6, 18, 14)); } + inline void Assembler::rett( Register s1, Register s2 ) { cti(); emit_int32( op(arith_op) | op3(rett_op3) | rs1(s1) | rs2(s2)); has_delay_slot(); } inline void Assembler::rett( Register s1, int simm13a, relocInfo::relocType rt) { cti(); emit_data( op(arith_op) | op3(rett_op3) | rs1(s1) | immed(true) | simm(simm13a, 13), rt); has_delay_slot(); } +inline void Assembler::save( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(save_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::save( Register s1, int simm13a, Register d ) { + // make sure frame is at least large enough for the register save area + assert(-simm13a >= 16 * wordSize, "frame too small"); + emit_int32( op(arith_op) | rd(d) | op3(save_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); +} + +inline void Assembler::restore( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(restore_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::restore( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(restore_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +// pp 216 + +inline void Assembler::saved() { v9_only(); emit_int32( op(arith_op) | fcn(0) | op3(saved_op3)); } +inline void Assembler::restored() { v9_only(); emit_int32( op(arith_op) | fcn(1) | op3(saved_op3)); } + inline void Assembler::sethi( int imm22a, Register d, RelocationHolder const& rspec ) { emit_data( op(branch_op) | rd(d) | op2(sethi_op2) | hi22(imm22a), rspec); } +inline void Assembler::sll( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(0) | rs2(s2) ); } +inline void Assembler::sll( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } +inline void Assembler::srl( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(0) | rs2(s2) ); } +inline void Assembler::srl( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } +inline void Assembler::sra( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(0) | rs2(s2) ); } +inline void Assembler::sra( Register s1, int imm5a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(0) | immed(true) | u_field(imm5a, 4, 0) ); } + +inline void Assembler::sllx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(1) | rs2(s2) ); } +inline void Assembler::sllx( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sll_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } +inline void Assembler::srlx( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(1) | rs2(s2) ); } +inline void Assembler::srlx( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(srl_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } +inline void Assembler::srax( Register s1, Register s2, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(1) | rs2(s2) ); } +inline void Assembler::srax( Register s1, int imm6a, Register d ) { v9_only(); emit_int32( op(arith_op) | rd(d) | op3(sra_op3) | rs1(s1) | sx(1) | immed(true) | u_field(imm6a, 5, 0) ); } + +inline void Assembler::sir( int simm13a ) { emit_int32( op(arith_op) | fcn(15) | op3(sir_op3) | immed(true) | simm(simm13a, 13)); } + + // pp 221 + +inline void Assembler::stbar() { emit_int32( op(arith_op) | op3(membar_op3) | u_field(15, 18, 14)); } + // pp 222 inline void Assembler::stf( FloatRegisterImpl::Width w, FloatRegister d, Register s1, Register s2) { emit_int32( op(ldst_op) | fd(d, w) | alt_op3(stf_op3, w) | rs1(s1) | rs2(s2) ); } @@ -120,6 +328,9 @@ inline void Assembler::stf( FloatRegisterImpl::Width w, FloatRegister d, Regi inline void Assembler::stxfsr( Register s1, Register s2) { v9_only(); emit_int32( op(ldst_op) | rd(G1) | op3(stfsr_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::stxfsr( Register s1, int simm13a) { v9_only(); emit_data( op(ldst_op) | rd(G1) | op3(stfsr_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, Register s2, int ia ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(stf_op3 | alt_bit_op3, w) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stfa( FloatRegisterImpl::Width w, FloatRegister d, Register s1, int simm13a ) { v9_only(); emit_int32( op(ldst_op) | fd(d, w) | alt_op3(stf_op3 | alt_bit_op3, w) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + // p 226 inline void Assembler::stb( Register d, Register s1, Register s2) { emit_int32( op(ldst_op) | rd(d) | op3(stb_op3) | rs1(s1) | rs2(s2) ); } @@ -135,9 +346,103 @@ inline void Assembler::stx( Register d, Register s1, int simm13a) { v9_only(); inline void Assembler::std( Register d, Register s1, Register s2) { v9_dep(); assert(d->is_even(), "not even"); emit_int32( op(ldst_op) | rd(d) | op3(std_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::std( Register d, Register s1, int simm13a) { v9_dep(); assert(d->is_even(), "not even"); emit_data( op(ldst_op) | rd(d) | op3(std_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::stba( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(stb_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stba( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(stb_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::stha( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(sth_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stha( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(sth_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::stwa( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(stw_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stwa( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(stw_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::stxa( Register d, Register s1, Register s2, int ia ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(stx_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stxa( Register d, Register s1, int simm13a ) { v9_only(); emit_int32( op(ldst_op) | rd(d) | op3(stx_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::stda( Register d, Register s1, Register s2, int ia ) { emit_int32( op(ldst_op) | rd(d) | op3(std_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::stda( Register d, Register s1, int simm13a ) { emit_int32( op(ldst_op) | rd(d) | op3(std_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +// pp 230 + +inline void Assembler::sub( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::sub( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +inline void Assembler::subcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 | cc_bit_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::subcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(sub_op3 | cc_bit_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::subc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::subc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } +inline void Assembler::subccc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 | cc_bit_op3) | rs1(s1) | rs2(s2) ); } +inline void Assembler::subccc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(subc_op3 | cc_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + // pp 231 inline void Assembler::swap( Register s1, Register s2, Register d) { v9_dep(); emit_int32( op(ldst_op) | rd(d) | op3(swap_op3) | rs1(s1) | rs2(s2) ); } inline void Assembler::swap( Register s1, int simm13a, Register d) { v9_dep(); emit_data( op(ldst_op) | rd(d) | op3(swap_op3) | rs1(s1) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::swapa( Register s1, Register s2, int ia, Register d ) { v9_dep(); emit_int32( op(ldst_op) | rd(d) | op3(swap_op3 | alt_bit_op3) | rs1(s1) | imm_asi(ia) | rs2(s2) ); } +inline void Assembler::swapa( Register s1, int simm13a, Register d ) { v9_dep(); emit_int32( op(ldst_op) | rd(d) | op3(swap_op3 | alt_bit_op3) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +// pp 234, note op in book is wrong, see pp 268 + +inline void Assembler::taddcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(taddcc_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::taddcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(taddcc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +// pp 235 + +inline void Assembler::tsubcc( Register s1, Register s2, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(tsubcc_op3 ) | rs1(s1) | rs2(s2) ); } +inline void Assembler::tsubcc( Register s1, int simm13a, Register d ) { emit_int32( op(arith_op) | rd(d) | op3(tsubcc_op3 ) | rs1(s1) | immed(true) | simm(simm13a, 13) ); } + +// pp 237 + +inline void Assembler::trap( Condition c, CC cc, Register s1, Register s2 ) { emit_int32( op(arith_op) | cond(c) | op3(trap_op3) | rs1(s1) | trapcc(cc) | rs2(s2)); } +inline void Assembler::trap( Condition c, CC cc, Register s1, int trapa ) { emit_int32( op(arith_op) | cond(c) | op3(trap_op3) | rs1(s1) | trapcc(cc) | immed(true) | u_field(trapa, 6, 0)); } +// simple uncond. trap +inline void Assembler::trap( int trapa ) { trap( always, icc, G0, trapa ); } + +inline void Assembler::wry(Register d) { v9_dep(); emit_int32(op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(0, 29, 25)); } +inline void Assembler::wrccr(Register s) { v9_only(); emit_int32(op(arith_op) | rs1(s) | op3(wrreg_op3) | u_field(2, 29, 25)); } +inline void Assembler::wrccr(Register s, int simm13a) { v9_only(); emit_int32(op(arith_op) | rs1(s) | op3(wrreg_op3) | u_field(2, 29, 25) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::wrasi(Register d) { v9_only(); emit_int32(op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(3, 29, 25)); } +// wrasi(d, imm) stores (d xor imm) to asi +inline void Assembler::wrasi(Register d, int simm13a) { v9_only(); emit_int32(op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(3, 29, 25) | immed(true) | simm(simm13a, 13)); } +inline void Assembler::wrfprs(Register d) { v9_only(); emit_int32(op(arith_op) | rs1(d) | op3(wrreg_op3) | u_field(6, 29, 25)); } + +inline void Assembler::alignaddr( Register s1, Register s2, Register d ) { vis1_only(); emit_int32( op(arith_op) | rd(d) | op3(alignaddr_op3) | rs1(s1) | opf(alignaddr_opf) | rs2(s2)); } + +inline void Assembler::faligndata( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(faligndata_op3) | fs1(s1, FloatRegisterImpl::D) | opf(faligndata_opf) | fs2(s2, FloatRegisterImpl::D)); } + +inline void Assembler::fzero( FloatRegisterImpl::Width w, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fzero_op3) | opf(0x62 - w)); } + +inline void Assembler::fsrc2( FloatRegisterImpl::Width w, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fsrc_op3) | opf(0x7A - w) | fs2(s2, w)); } + +inline void Assembler::fnot1( FloatRegisterImpl::Width w, FloatRegister s1, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, w) | op3(fnot_op3) | fs1(s1, w) | opf(0x6C - w)); } + +inline void Assembler::fpmerge( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis1_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(0x36) | fs1(s1, FloatRegisterImpl::S) | opf(0x4b) | fs2(s2, FloatRegisterImpl::S)); } + +inline void Assembler::stpartialf( Register s1, Register s2, FloatRegister d, int ia ) { vis1_only(); emit_int32( op(ldst_op) | fd(d, FloatRegisterImpl::D) | op3(stpartialf_op3) | rs1(s1) | imm_asi(ia) | rs2(s2)); } + +// VIS2 instructions + +inline void Assembler::edge8n( Register s1, Register s2, Register d ) { vis2_only(); emit_int32( op(arith_op) | rd(d) | op3(edge_op3) | rs1(s1) | opf(edge8n_opf) | rs2(s2)); } + +inline void Assembler::bmask( Register s1, Register s2, Register d ) { vis2_only(); emit_int32( op(arith_op) | rd(d) | op3(bmask_op3) | rs1(s1) | opf(bmask_opf) | rs2(s2)); } +inline void Assembler::bshuffle( FloatRegister s1, FloatRegister s2, FloatRegister d ) { vis2_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(bshuffle_op3) | fs1(s1, FloatRegisterImpl::D) | opf(bshuffle_opf) | fs2(s2, FloatRegisterImpl::D)); } + +// VIS3 instructions + +inline void Assembler::movstosw( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mstosw_opf) | fs2(s, FloatRegisterImpl::S)); } +inline void Assembler::movstouw( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mstouw_opf) | fs2(s, FloatRegisterImpl::S)); } +inline void Assembler::movdtox( FloatRegister s, Register d ) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(mftoi_op3) | opf(mdtox_opf) | fs2(s, FloatRegisterImpl::D)); } + +inline void Assembler::movwtos( Register s, FloatRegister d ) { vis3_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::S) | op3(mftoi_op3) | opf(mwtos_opf) | rs2(s)); } +inline void Assembler::movxtod( Register s, FloatRegister d ) { vis3_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(mftoi_op3) | opf(mxtod_opf) | rs2(s)); } + +inline void Assembler::xmulx(Register s1, Register s2, Register d) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(xmulx_op3) | rs1(s1) | opf(xmulx_opf) | rs2(s2)); } +inline void Assembler::xmulxhi(Register s1, Register s2, Register d) { vis3_only(); emit_int32( op(arith_op) | rd(d) | op3(xmulx_op3) | rs1(s1) | opf(xmulxhi_opf) | rs2(s2)); } + +// Crypto SHA instructions + +inline void Assembler::sha1() { sha1_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha1_opf)); } +inline void Assembler::sha256() { sha256_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha256_opf)); } +inline void Assembler::sha512() { sha512_only(); emit_int32( op(arith_op) | op3(sha_op3) | opf(sha512_opf)); } + +// CRC32C instruction + +inline void Assembler::crc32c( FloatRegister s1, FloatRegister s2, FloatRegister d ) { crc32c_only(); emit_int32( op(arith_op) | fd(d, FloatRegisterImpl::D) | op3(crc32c_op3) | fs1(s1, FloatRegisterImpl::D) | opf(crc32c_opf) | fs2(s2, FloatRegisterImpl::D)); } + #endif // CPU_SPARC_VM_ASSEMBLER_SPARC_INLINE_HPP diff --git a/hotspot/src/cpu/sparc/vm/globalDefinitions_sparc.hpp b/hotspot/src/cpu/sparc/vm/globalDefinitions_sparc.hpp index 17492911197..ddda08068bd 100644 --- a/hotspot/src/cpu/sparc/vm/globalDefinitions_sparc.hpp +++ b/hotspot/src/cpu/sparc/vm/globalDefinitions_sparc.hpp @@ -30,6 +30,10 @@ const int BytesPerInstWord = 4; const int StackAlignmentInBytes = (2*wordSize); +// Indicates whether the C calling conventions require that +// 32-bit integer argument values are extended to 64 bits. +const bool CCallingConventionRequiresIntsAsLongs = false; + #define SUPPORTS_NATIVE_CX8 // The expected size in bytes of a cache line, used to pad data structures. diff --git a/hotspot/src/cpu/sparc/vm/jvmciCodeInstaller_sparc.cpp b/hotspot/src/cpu/sparc/vm/jvmciCodeInstaller_sparc.cpp index c50844963d0..50431ccca89 100644 --- a/hotspot/src/cpu/sparc/vm/jvmciCodeInstaller_sparc.cpp +++ b/hotspot/src/cpu/sparc/vm/jvmciCodeInstaller_sparc.cpp @@ -73,7 +73,7 @@ void CodeInstaller::pd_patch_MetaspaceConstant(int pc_offset, Handle constant, T NativeMovConstReg32* move = nativeMovConstReg32_at(pc); narrowKlass narrowOop = record_narrow_metadata_reference(constant, CHECK); move->set_data((intptr_t)narrowOop); - TRACE_jvmci_3("relocating (narrow metaspace constant) at %p/%p", pc, narrowOop); + TRACE_jvmci_3("relocating (narrow metaspace constant) at " PTR_FORMAT "/0x%x", p2i(pc), narrowOop); #else JVMCI_ERROR("compressed Klass* on 32bit"); #endif @@ -81,7 +81,7 @@ void CodeInstaller::pd_patch_MetaspaceConstant(int pc_offset, Handle constant, T NativeMovConstReg* move = nativeMovConstReg_at(pc); Metadata* reference = record_metadata_reference(constant, CHECK); move->set_data((intptr_t)reference); - TRACE_jvmci_3("relocating (metaspace constant) at %p/%p", pc, reference); + TRACE_jvmci_3("relocating (metaspace constant) at " PTR_FORMAT "/" PTR_FORMAT, p2i(pc), p2i(reference)); } } diff --git a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.cpp b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.cpp index 0920bc90e08..b8b1bbf9ddf 100644 --- a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.cpp +++ b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.cpp @@ -181,19 +181,6 @@ void MacroAssembler::null_check(Register reg, int offset) { // Ring buffer jumps -#ifndef PRODUCT -void MacroAssembler::ret( bool trace ) { if (trace) { - mov(I7, O7); // traceable register - JMP(O7, 2 * BytesPerInstWord); - } else { - jmpl( I7, 2 * BytesPerInstWord, G0 ); - } - } - -void MacroAssembler::retl( bool trace ) { if (trace) JMP(O7, 2 * BytesPerInstWord); - else jmpl( O7, 2 * BytesPerInstWord, G0 ); } -#endif /* PRODUCT */ - void MacroAssembler::jmp2(Register r1, Register r2, const char* file, int line ) { assert_not_delayed(); @@ -758,8 +745,8 @@ void MacroAssembler::set_vm_result(Register oop_result) { } -void MacroAssembler::ic_call(address entry, bool emit_delay) { - RelocationHolder rspec = virtual_call_Relocation::spec(pc()); +void MacroAssembler::ic_call(address entry, bool emit_delay, jint method_index) { + RelocationHolder rspec = virtual_call_Relocation::spec(pc(), method_index); patchable_set((intptr_t)Universe::non_oop_word(), G5_inline_cache_reg); relocate(rspec); call(entry, relocInfo::none); @@ -768,7 +755,6 @@ void MacroAssembler::ic_call(address entry, bool emit_delay) { } } - void MacroAssembler::card_table_write(jbyte* byte_map_base, Register tmp, Register obj) { #ifdef _LP64 diff --git a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.hpp b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.hpp index f2757fbe4f5..eb6188a3b9a 100644 --- a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.hpp +++ b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.hpp @@ -680,8 +680,8 @@ class MacroAssembler : public Assembler { inline int get_pc( Register d ); // Sparc shorthands(pp 85, V8 manual, pp 289 V9 manual) - inline void cmp( Register s1, Register s2 ) { subcc( s1, s2, G0 ); } - inline void cmp( Register s1, int simm13a ) { subcc( s1, simm13a, G0 ); } + inline void cmp( Register s1, Register s2 ); + inline void cmp( Register s1, int simm13a ); inline void jmp( Register s1, Register s2 ); inline void jmp( Register s1, int simm13a, RelocationHolder const& rspec = RelocationHolder() ); @@ -689,7 +689,11 @@ class MacroAssembler : public Assembler { // Check if the call target is out of wdisp30 range (relative to the code cache) static inline bool is_far_target(address d); inline void call( address d, relocInfo::relocType rt = relocInfo::runtime_call_type ); + inline void call( address d, RelocationHolder const& rspec); + inline void call( Label& L, relocInfo::relocType rt = relocInfo::runtime_call_type ); + inline void call( Label& L, RelocationHolder const& rspec); + inline void callr( Register s1, Register s2 ); inline void callr( Register s1, int simm13a, RelocationHolder const& rspec = RelocationHolder() ); @@ -697,23 +701,10 @@ class MacroAssembler : public Assembler { inline void iprefetch( address d, relocInfo::relocType rt = relocInfo::none ); inline void iprefetch( Label& L); - inline void tst( Register s ) { orcc( G0, s, G0 ); } + inline void tst( Register s ); -#ifdef PRODUCT - inline void ret( bool trace = TraceJumps ) { if (trace) { - mov(I7, O7); // traceable register - JMP(O7, 2 * BytesPerInstWord); - } else { - jmpl( I7, 2 * BytesPerInstWord, G0 ); - } - } - - inline void retl( bool trace = TraceJumps ) { if (trace) JMP(O7, 2 * BytesPerInstWord); - else jmpl( O7, 2 * BytesPerInstWord, G0 ); } -#else - void ret( bool trace = TraceJumps ); - void retl( bool trace = TraceJumps ); -#endif /* PRODUCT */ + inline void ret( bool trace = TraceJumps ); + inline void retl( bool trace = TraceJumps ); // Required platform-specific helpers for Label::patch_instructions. // They _shadow_ the declarations in AbstractAssembler, which are undefined. @@ -746,26 +737,20 @@ public: static int insts_for_set64(jlong value); // sign-extend 32 to 64 - inline void signx( Register s, Register d ) { sra( s, G0, d); } - inline void signx( Register d ) { sra( d, G0, d); } + inline void signx( Register s, Register d ); + inline void signx( Register d ); - inline void not1( Register s, Register d ) { xnor( s, G0, d ); } - inline void not1( Register d ) { xnor( d, G0, d ); } + inline void not1( Register s, Register d ); + inline void not1( Register d ); - inline void neg( Register s, Register d ) { sub( G0, s, d ); } - inline void neg( Register d ) { sub( G0, d, d ); } + inline void neg( Register s, Register d ); + inline void neg( Register d ); - inline void cas( Register s1, Register s2, Register d) { casa( s1, s2, d, ASI_PRIMARY); } - inline void casx( Register s1, Register s2, Register d) { casxa(s1, s2, d, ASI_PRIMARY); } + inline void cas( Register s1, Register s2, Register d); + inline void casx( Register s1, Register s2, Register d); // Functions for isolating 64 bit atomic swaps for LP64 // cas_ptr will perform cas for 32 bit VM's and casx for 64 bit VM's - inline void cas_ptr( Register s1, Register s2, Register d) { -#ifdef _LP64 - casx( s1, s2, d ); -#else - cas( s1, s2, d ); -#endif - } + inline void cas_ptr( Register s1, Register s2, Register d); // Functions for isolating 64 bit shifts for LP64 inline void sll_ptr( Register s1, Register s2, Register d ); @@ -775,14 +760,14 @@ public: inline void srl_ptr( Register s1, int imm6a, Register d ); // little-endian - inline void casl( Register s1, Register s2, Register d) { casa( s1, s2, d, ASI_PRIMARY_LITTLE); } - inline void casxl( Register s1, Register s2, Register d) { casxa(s1, s2, d, ASI_PRIMARY_LITTLE); } + inline void casl( Register s1, Register s2, Register d); + inline void casxl( Register s1, Register s2, Register d); - inline void inc( Register d, int const13 = 1 ) { add( d, const13, d); } - inline void inccc( Register d, int const13 = 1 ) { addcc( d, const13, d); } + inline void inc( Register d, int const13 = 1 ); + inline void inccc( Register d, int const13 = 1 ); - inline void dec( Register d, int const13 = 1 ) { sub( d, const13, d); } - inline void deccc( Register d, int const13 = 1 ) { subcc( d, const13, d); } + inline void dec( Register d, int const13 = 1 ); + inline void deccc( Register d, int const13 = 1 ); using Assembler::add; inline void add(Register s1, int simm13a, Register d, relocInfo::relocType rtype); @@ -793,19 +778,19 @@ public: using Assembler::andn; inline void andn( Register s1, RegisterOrConstant s2, Register d); - inline void btst( Register s1, Register s2 ) { andcc( s1, s2, G0 ); } - inline void btst( int simm13a, Register s ) { andcc( s, simm13a, G0 ); } + inline void btst( Register s1, Register s2 ); + inline void btst( int simm13a, Register s ); - inline void bset( Register s1, Register s2 ) { or3( s1, s2, s2 ); } - inline void bset( int simm13a, Register s ) { or3( s, simm13a, s ); } + inline void bset( Register s1, Register s2 ); + inline void bset( int simm13a, Register s ); - inline void bclr( Register s1, Register s2 ) { andn( s1, s2, s2 ); } - inline void bclr( int simm13a, Register s ) { andn( s, simm13a, s ); } + inline void bclr( Register s1, Register s2 ); + inline void bclr( int simm13a, Register s ); - inline void btog( Register s1, Register s2 ) { xor3( s1, s2, s2 ); } - inline void btog( int simm13a, Register s ) { xor3( s, simm13a, s ); } + inline void btog( Register s1, Register s2 ); + inline void btog( int simm13a, Register s ); - inline void clr( Register d ) { or3( G0, G0, d ); } + inline void clr( Register d ); inline void clrb( Register s1, Register s2); inline void clrh( Register s1, Register s2); @@ -818,9 +803,9 @@ public: inline void clrx( Register s1, int simm13a); // copy & clear upper word - inline void clruw( Register s, Register d ) { srl( s, G0, d); } + inline void clruw( Register s, Register d ); // clear upper word - inline void clruwu( Register d ) { srl( d, G0, d); } + inline void clruwu( Register d ); using Assembler::ldsb; using Assembler::ldsh; @@ -864,10 +849,10 @@ public: inline void ldf(FloatRegisterImpl::Width w, const Address& a, FloatRegister d, int offset = 0); // little-endian - inline void lduwl(Register s1, Register s2, Register d) { lduwa(s1, s2, ASI_PRIMARY_LITTLE, d); } - inline void ldswl(Register s1, Register s2, Register d) { ldswa(s1, s2, ASI_PRIMARY_LITTLE, d);} - inline void ldxl( Register s1, Register s2, Register d) { ldxa(s1, s2, ASI_PRIMARY_LITTLE, d); } - inline void ldfl(FloatRegisterImpl::Width w, Register s1, Register s2, FloatRegister d) { ldfa(w, s1, s2, ASI_PRIMARY_LITTLE, d); } + inline void lduwl(Register s1, Register s2, Register d); + inline void ldswl(Register s1, Register s2, Register d); + inline void ldxl( Register s1, Register s2, Register d); + inline void ldfl(FloatRegisterImpl::Width w, Register s1, Register s2, FloatRegister d); // membar psuedo instruction. takes into account target memory model. inline void membar( Assembler::Membar_mask_bits const7a ); @@ -876,17 +861,11 @@ public: inline bool membar_has_effect( Assembler::Membar_mask_bits const7a ); // mov pseudo instructions - inline void mov( Register s, Register d) { - if ( s != d ) or3( G0, s, d); - else assert_not_delayed(); // Put something useful in the delay slot! - } + inline void mov( Register s, Register d); - inline void mov_or_nop( Register s, Register d) { - if ( s != d ) or3( G0, s, d); - else nop(); - } + inline void mov_or_nop( Register s, Register d); - inline void mov( int simm13a, Register d) { or3( G0, simm13a, d); } + inline void mov( int simm13a, Register d); using Assembler::prefetch; inline void prefetch(const Address& a, PrefetchFcn F, int offset = 0); @@ -961,11 +940,7 @@ public: // handy macros: - inline void round_to( Register r, int modulus ) { - assert_not_delayed(); - inc( r, modulus - 1 ); - and3( r, -modulus, r ); - } + inline void round_to( Register r, int modulus ); // -------------------------------------------------- @@ -1033,9 +1008,9 @@ public: // These are idioms to flag the need for care with accessing bools but on // this platform we assume byte size - inline void stbool(Register d, const Address& a) { stb(d, a); } - inline void ldbool(const Address& a, Register d) { ldub(a, d); } - inline void movbool( bool boolconst, Register d) { mov( (int) boolconst, d); } + inline void stbool(Register d, const Address& a); + inline void ldbool(const Address& a, Register d); + inline void movbool( bool boolconst, Register d); // klass oop manipulations if compressed void load_klass(Register src_oop, Register klass); @@ -1106,7 +1081,7 @@ public: void set_vm_result(Register oop_result); // Emit the CompiledIC call idiom - void ic_call(address entry, bool emit_delay = true); + void ic_call(address entry, bool emit_delay = true, jint method_index = 0); // if call_VM_base was called with check_exceptions=false, then call // check_and_forward_exception to handle exceptions when it is safe @@ -1371,12 +1346,7 @@ public: // Stack overflow checking // Note: this clobbers G3_scratch - void bang_stack_with_offset(int offset) { - // stack grows down, caller passes positive offset - assert(offset > 0, "must bang with negative offset"); - set((-offset)+STACK_BIAS, G3_scratch); - st(G0, SP, G3_scratch); - } + inline void bang_stack_with_offset(int offset); // Writes to stack successive pages until offset reached to check for // stack overflow + shadow pages. Clobbers tsp and scratch registers. diff --git a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.inline.hpp b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.inline.hpp index 3518d0b6336..2f1c949bb7f 100644 --- a/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.inline.hpp +++ b/hotspot/src/cpu/sparc/vm/macroAssembler_sparc.inline.hpp @@ -187,6 +187,33 @@ inline void MacroAssembler::st_long( Register d, const Address& a, int offset ) #endif } +inline void MacroAssembler::stbool(Register d, const Address& a) { stb(d, a); } +inline void MacroAssembler::ldbool(const Address& a, Register d) { ldub(a, d); } +inline void MacroAssembler::movbool( bool boolconst, Register d) { mov( (int) boolconst, d); } + + +inline void MacroAssembler::signx( Register s, Register d ) { sra( s, G0, d); } +inline void MacroAssembler::signx( Register d ) { sra( d, G0, d); } + +inline void MacroAssembler::not1( Register s, Register d ) { xnor( s, G0, d ); } +inline void MacroAssembler::not1( Register d ) { xnor( d, G0, d ); } + +inline void MacroAssembler::neg( Register s, Register d ) { sub( G0, s, d ); } +inline void MacroAssembler::neg( Register d ) { sub( G0, d, d ); } + +inline void MacroAssembler::cas( Register s1, Register s2, Register d) { casa( s1, s2, d, ASI_PRIMARY); } +inline void MacroAssembler::casx( Register s1, Register s2, Register d) { casxa(s1, s2, d, ASI_PRIMARY); } + +// Functions for isolating 64 bit atomic swaps for LP64 +// cas_ptr will perform cas for 32 bit VM's and casx for 64 bit VM's +inline void MacroAssembler::cas_ptr( Register s1, Register s2, Register d) { +#ifdef _LP64 + casx( s1, s2, d ); +#else + cas( s1, s2, d ); +#endif +} + // Functions for isolating 64 bit shifts for LP64 inline void MacroAssembler::sll_ptr( Register s1, Register s2, Register d ) { @@ -226,6 +253,15 @@ inline void MacroAssembler::sll_ptr( Register s1, RegisterOrConstant s2, Registe else sll_ptr(s1, s2.as_constant(), d); } +inline void MacroAssembler::casl( Register s1, Register s2, Register d) { casa( s1, s2, d, ASI_PRIMARY_LITTLE); } +inline void MacroAssembler::casxl( Register s1, Register s2, Register d) { casxa(s1, s2, d, ASI_PRIMARY_LITTLE); } + +inline void MacroAssembler::inc( Register d, int const13 ) { add( d, const13, d); } +inline void MacroAssembler::inccc( Register d, int const13 ) { addcc( d, const13, d); } + +inline void MacroAssembler::dec( Register d, int const13 ) { sub( d, const13, d); } +inline void MacroAssembler::deccc( Register d, int const13 ) { subcc( d, const13, d); } + // Use the right branch for the platform inline void MacroAssembler::br( Condition c, bool a, Predict p, address d, relocInfo::relocType rt ) { @@ -298,6 +334,10 @@ inline bool MacroAssembler::is_far_target(address d) { // expense of relocation and if we overflow the displacement // of the quick call instruction. inline void MacroAssembler::call( address d, relocInfo::relocType rt ) { + MacroAssembler::call(d, Relocation::spec_simple(rt)); +} + +inline void MacroAssembler::call( address d, RelocationHolder const& rspec ) { #ifdef _LP64 intptr_t disp; // NULL is ok because it will be relocated later. @@ -309,14 +349,14 @@ inline void MacroAssembler::call( address d, relocInfo::relocType rt ) { // Is this address within range of the call instruction? // If not, use the expensive instruction sequence if (is_far_target(d)) { - relocate(rt); + relocate(rspec); AddressLiteral dest(d); jumpl_to(dest, O7, O7); } else { - Assembler::call(d, rt); + Assembler::call(d, rspec); } #else - Assembler::call( d, rt ); + Assembler::call( d, rspec ); #endif } @@ -337,6 +377,24 @@ inline void MacroAssembler::iprefetch( address d, relocInfo::relocType rt ) { } inline void MacroAssembler::iprefetch( Label& L) { iprefetch( target(L) ); } +inline void MacroAssembler::tst( Register s ) { orcc( G0, s, G0 ); } + +inline void MacroAssembler::ret( bool trace ) { + if (trace) { + mov(I7, O7); // traceable register + JMP(O7, 2 * BytesPerInstWord); + } else { + jmpl( I7, 2 * BytesPerInstWord, G0 ); + } +} + +inline void MacroAssembler::retl( bool trace ) { + if (trace) { + JMP(O7, 2 * BytesPerInstWord); + } else { + jmpl( O7, 2 * BytesPerInstWord, G0 ); + } +} // clobbers o7 on V8!! // returns delta from gotten pc to addr after @@ -346,6 +404,8 @@ inline int MacroAssembler::get_pc( Register d ) { return offset() - x; } +inline void MacroAssembler::cmp( Register s1, Register s2 ) { subcc( s1, s2, G0 ); } +inline void MacroAssembler::cmp( Register s1, int simm13a ) { subcc( s1, simm13a, G0 ); } // Note: All MacroAssembler::set_foo functions are defined out-of-line. @@ -521,6 +581,12 @@ inline void MacroAssembler::store_long_argument( Register s, Argument& a ) { } #endif +inline void MacroAssembler::round_to( Register r, int modulus ) { + assert_not_delayed(); + inc( r, modulus - 1 ); + and3( r, -modulus, r ); +} + inline void MacroAssembler::add(Register s1, int simm13a, Register d, relocInfo::relocType rtype) { relocate(rtype); add(s1, simm13a, d); @@ -547,6 +613,20 @@ inline void MacroAssembler::andn(Register s1, RegisterOrConstant s2, Register d) else andn(s1, s2.as_constant(), d); } +inline void MacroAssembler::btst( Register s1, Register s2 ) { andcc( s1, s2, G0 ); } +inline void MacroAssembler::btst( int simm13a, Register s ) { andcc( s, simm13a, G0 ); } + +inline void MacroAssembler::bset( Register s1, Register s2 ) { or3( s1, s2, s2 ); } +inline void MacroAssembler::bset( int simm13a, Register s ) { or3( s, simm13a, s ); } + +inline void MacroAssembler::bclr( Register s1, Register s2 ) { andn( s1, s2, s2 ); } +inline void MacroAssembler::bclr( int simm13a, Register s ) { andn( s, simm13a, s ); } + +inline void MacroAssembler::btog( Register s1, Register s2 ) { xor3( s1, s2, s2 ); } +inline void MacroAssembler::btog( int simm13a, Register s ) { xor3( s, simm13a, s ); } + +inline void MacroAssembler::clr( Register d ) { or3( G0, G0, d ); } + inline void MacroAssembler::clrb( Register s1, Register s2) { stb( G0, s1, s2 ); } inline void MacroAssembler::clrh( Register s1, Register s2) { sth( G0, s1, s2 ); } inline void MacroAssembler::clr( Register s1, Register s2) { stw( G0, s1, s2 ); } @@ -557,6 +637,9 @@ inline void MacroAssembler::clrh( Register s1, int simm13a) { sth( G0, s1, simm1 inline void MacroAssembler::clr( Register s1, int simm13a) { stw( G0, s1, simm13a); } inline void MacroAssembler::clrx( Register s1, int simm13a) { stx( G0, s1, simm13a); } +inline void MacroAssembler::clruw( Register s, Register d ) { srl( s, G0, d); } +inline void MacroAssembler::clruwu( Register d ) { srl( d, G0, d); } + #ifdef _LP64 // Make all 32 bit loads signed so 64 bit registers maintain proper sign inline void MacroAssembler::ld( Register s1, Register s2, Register d) { ldsw( s1, s2, d); } @@ -638,6 +721,11 @@ inline void MacroAssembler::ldf(FloatRegisterImpl::Width w, const Address& a, Fl } } +inline void MacroAssembler::lduwl(Register s1, Register s2, Register d) { lduwa(s1, s2, ASI_PRIMARY_LITTLE, d); } +inline void MacroAssembler::ldswl(Register s1, Register s2, Register d) { ldswa(s1, s2, ASI_PRIMARY_LITTLE, d);} +inline void MacroAssembler::ldxl( Register s1, Register s2, Register d) { ldxa(s1, s2, ASI_PRIMARY_LITTLE, d); } +inline void MacroAssembler::ldfl(FloatRegisterImpl::Width w, Register s1, Register s2, FloatRegister d) { ldfa(w, s1, s2, ASI_PRIMARY_LITTLE, d); } + // returns if membar generates anything, obviously this code should mirror // membar below. inline bool MacroAssembler::membar_has_effect( Membar_mask_bits const7a ) { @@ -664,6 +752,24 @@ inline void MacroAssembler::membar( Membar_mask_bits const7a ) { } } +inline void MacroAssembler::mov(Register s, Register d) { + if (s != d) { + or3(G0, s, d); + } else { + assert_not_delayed(); // Put something useful in the delay slot! + } +} + +inline void MacroAssembler::mov_or_nop(Register s, Register d) { + if (s != d) { + or3(G0, s, d); + } else { + nop(); + } +} + +inline void MacroAssembler::mov( int simm13a, Register d) { or3( G0, simm13a, d); } + inline void MacroAssembler::prefetch(const Address& a, PrefetchFcn f, int offset) { relocate(a.rspec(offset)); assert(!a.has_index(), ""); @@ -734,4 +840,11 @@ inline void MacroAssembler::swap(const Address& a, Register d, int offset) { else { swap(a.base(), a.disp() + offset, d); } } +inline void MacroAssembler::bang_stack_with_offset(int offset) { + // stack grows down, caller passes positive offset + assert(offset > 0, "must bang with negative offset"); + set((-offset)+STACK_BIAS, G3_scratch); + st(G0, SP, G3_scratch); +} + #endif // CPU_SPARC_VM_MACROASSEMBLER_SPARC_INLINE_HPP diff --git a/hotspot/src/cpu/sparc/vm/nativeInst_sparc.cpp b/hotspot/src/cpu/sparc/vm/nativeInst_sparc.cpp index 024049ca5b2..d17550859c0 100644 --- a/hotspot/src/cpu/sparc/vm/nativeInst_sparc.cpp +++ b/hotspot/src/cpu/sparc/vm/nativeInst_sparc.cpp @@ -131,8 +131,9 @@ bool NativeInstruction::is_load_store_with_small_offset(Register reg) { void NativeCall::verify() { NativeInstruction::verify(); // make sure code pattern is actually a call instruction - if (!is_op(long_at(0), Assembler::call_op)) { - fatal("not a call"); + int x = long_at(0); + if (!is_op(x, Assembler::call_op)) { + fatal("not a call: 0x%x @ " INTPTR_FORMAT, x, p2i(instruction_address())); } } diff --git a/hotspot/src/cpu/sparc/vm/sparc.ad b/hotspot/src/cpu/sparc/vm/sparc.ad index 336a3cdd19e..54569cde677 100644 --- a/hotspot/src/cpu/sparc/vm/sparc.ad +++ b/hotspot/src/cpu/sparc/vm/sparc.ad @@ -1001,7 +1001,7 @@ void emit_form3_mem_reg(CodeBuffer &cbuf, PhaseRegAlloc* ra, const MachNode* n, #endif } -void emit_call_reloc(CodeBuffer &cbuf, intptr_t entry_point, relocInfo::relocType rtype, bool preserve_g2 = false) { +void emit_call_reloc(CodeBuffer &cbuf, intptr_t entry_point, RelocationHolder const& rspec, bool preserve_g2 = false) { // The method which records debug information at every safepoint // expects the call to be the first instruction in the snippet as // it creates a PcDesc structure which tracks the offset of a call @@ -1023,7 +1023,7 @@ void emit_call_reloc(CodeBuffer &cbuf, intptr_t entry_point, relocInfo::relocTyp int startpos = __ offset(); #endif /* ASSERT */ - __ call((address)entry_point, rtype); + __ call((address)entry_point, rspec); if (preserve_g2) __ delayed()->mov(G2, L7); else __ delayed()->nop(); @@ -2598,8 +2598,7 @@ encode %{ enc_class Java_To_Runtime (method meth) %{ // CALL Java_To_Runtime // CALL directly to the runtime // The user of this is responsible for ensuring that R_L7 is empty (killed). - emit_call_reloc(cbuf, $meth$$method, relocInfo::runtime_call_type, - /*preserve_g2=*/true); + emit_call_reloc(cbuf, $meth$$method, runtime_call_Relocation::spec(), /*preserve_g2=*/true); %} enc_class preserve_SP %{ @@ -2616,13 +2615,14 @@ encode %{ // CALL to fixup routine. Fixup routine uses ScopeDesc info to determine // who we intended to call. if (!_method) { - emit_call_reloc(cbuf, $meth$$method, relocInfo::runtime_call_type); - } else if (_optimized_virtual) { - emit_call_reloc(cbuf, $meth$$method, relocInfo::opt_virtual_call_type); + emit_call_reloc(cbuf, $meth$$method, runtime_call_Relocation::spec()); } else { - emit_call_reloc(cbuf, $meth$$method, relocInfo::static_call_type); - } - if (_method) { // Emit stub for static call. + int method_index = resolved_method_index(cbuf); + RelocationHolder rspec = _optimized_virtual ? opt_virtual_call_Relocation::spec(method_index) + : static_call_Relocation::spec(method_index); + emit_call_reloc(cbuf, $meth$$method, rspec); + + // Emit stub for static call. address stub = CompiledStaticCall::emit_to_interp_stub(cbuf); // Stub does not fit into scratch buffer if TraceJumps is enabled if (stub == NULL && !(TraceJumps && Compile::current()->in_scratch_emit_size())) { @@ -2643,7 +2643,7 @@ encode %{ Register G5_ic_reg = reg_to_register_object(Matcher::inline_cache_reg_encode()); assert(G5_ic_reg == G5_inline_cache_reg, "G5_inline_cache_reg used in assemble_ic_buffer_code()"); assert(G5_ic_reg == G5_megamorphic_method, "G5_megamorphic_method used in megamorphic call stub"); - __ ic_call((address)$meth$$method); + __ ic_call((address)$meth$$method, /*emit_delay=*/true, resolved_method_index(cbuf)); } else { assert(!UseInlineCaches, "expect vtable calls only if not using ICs"); // Just go thru the vtable @@ -10074,10 +10074,10 @@ instruct string_compareL(o0RegP str1, o1RegP str2, g3RegI cnt1, g4RegI cnt2, not format %{ "String Compare byte[] $str1,$cnt1,$str2,$cnt2 -> $result // KILL $tmp" %} ins_encode %{ __ string_compare($str1$$Register, $str2$$Register, - $cnt1$$Register, $cnt2$$Register, + $cnt1$$Register, $cnt2$$Register, $tmp$$Register, $tmp$$Register, $result$$Register, StrIntrinsicNode::LL); - %} + %} ins_pipe(long_memory_op); %} @@ -10093,7 +10093,7 @@ instruct string_compareU(o0RegP str1, o1RegP str2, g3RegI cnt1, g4RegI cnt2, not $cnt1$$Register, $cnt2$$Register, $tmp$$Register, $tmp$$Register, $result$$Register, StrIntrinsicNode::UU); - %} + %} ins_pipe(long_memory_op); %} @@ -10109,7 +10109,7 @@ instruct string_compareLU(o0RegP str1, o1RegP str2, g3RegI cnt1, g4RegI cnt2, no $cnt1$$Register, $cnt2$$Register, $tmp1$$Register, $tmp2$$Register, $result$$Register, StrIntrinsicNode::LU); - %} + %} ins_pipe(long_memory_op); %} @@ -10122,10 +10122,10 @@ instruct string_compareUL(o0RegP str1, o1RegP str2, g3RegI cnt1, g4RegI cnt2, no format %{ "String Compare byte[] $str1,$cnt1,$str2,$cnt2 -> $result // KILL $tmp1,$tmp2" %} ins_encode %{ __ string_compare($str2$$Register, $str1$$Register, - $cnt2$$Register, $cnt1$$Register, + $cnt2$$Register, $cnt1$$Register, $tmp1$$Register, $tmp2$$Register, $result$$Register, StrIntrinsicNode::UL); - %} + %} ins_pipe(long_memory_op); %} diff --git a/hotspot/src/cpu/sparc/vm/vmStructs_sparc.hpp b/hotspot/src/cpu/sparc/vm/vmStructs_sparc.hpp index 24b008b4243..c8a8daf2844 100644 --- a/hotspot/src/cpu/sparc/vm/vmStructs_sparc.hpp +++ b/hotspot/src/cpu/sparc/vm/vmStructs_sparc.hpp @@ -1,5 +1,5 @@ /* - * Copyright (c) 2001, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2001, 2015, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -29,21 +29,12 @@ // constants required by the Serviceability Agent. This file is // referenced by vmStructs.cpp. -#define VM_STRUCTS_CPU(nonstatic_field, static_field, unchecked_nonstatic_field, volatile_nonstatic_field, nonproduct_nonstatic_field, c2_nonstatic_field, unchecked_c1_static_field, unchecked_c2_static_field) \ - \ - /******************************/ \ - /* JavaCallWrapper */ \ - /******************************/ \ - /******************************/ \ - /* JavaFrameAnchor */ \ - /******************************/ \ - volatile_nonstatic_field(JavaFrameAnchor, _flags, int) \ - static_field(VM_Version, _features, int) +#define VM_STRUCTS_CPU(nonstatic_field, static_field, unchecked_nonstatic_field, volatile_nonstatic_field, nonproduct_nonstatic_field, c2_nonstatic_field, unchecked_c1_static_field, unchecked_c2_static_field) \ + volatile_nonstatic_field(JavaFrameAnchor, _flags, int) #define VM_TYPES_CPU(declare_type, declare_toplevel_type, declare_oop_type, declare_integer_type, declare_unsigned_integer_type, declare_c1_toplevel_type, declare_c2_type, declare_c2_toplevel_type) \ - declare_toplevel_type(VM_Version) -#define VM_INT_CONSTANTS_CPU(declare_constant, declare_preprocessor_constant, declare_c1_constant, declare_c2_constant, declare_c2_preprocessor_constant) \ +#define VM_INT_CONSTANTS_CPU(declare_constant, declare_preprocessor_constant, declare_c1_constant, declare_c2_constant, declare_c2_preprocessor_constant) \ /******************************/ \ /* Register numbers (C2 only) */ \ /******************************/ \ diff --git a/hotspot/src/cpu/sparc/vm/vm_version_sparc.cpp b/hotspot/src/cpu/sparc/vm/vm_version_sparc.cpp index 6891ca56998..fe7ce4f315e 100644 --- a/hotspot/src/cpu/sparc/vm/vm_version_sparc.cpp +++ b/hotspot/src/cpu/sparc/vm/vm_version_sparc.cpp @@ -30,13 +30,10 @@ #include "runtime/stubCodeGenerator.hpp" #include "vm_version_sparc.hpp" -int VM_Version::_features = VM_Version::unknown_m; -const char* VM_Version::_features_str = ""; unsigned int VM_Version::_L2_data_cache_line_size = 0; void VM_Version::initialize() { - - assert(_features != VM_Version::unknown_m, "System pre-initialization is not complete."); + assert(_features != 0, "System pre-initialization is not complete."); guarantee(VM_Version::has_v9(), "only SPARC v9 is supported"); PrefetchCopyIntervalInBytes = prefetch_copy_interval_in_bytes(); @@ -214,7 +211,7 @@ void VM_Version::initialize() { (!has_hardware_fsmuld() ? ", no-fsmuld" : "")); // buf is started with ", " or is empty - _features_str = os::strdup(strlen(buf) > 2 ? buf + 2 : buf); + _features_string = os::strdup(strlen(buf) > 2 ? buf + 2 : buf); // UseVIS is set to the smallest of what hardware supports and what // the command line requires. I.e., you cannot set UseVIS to 3 on @@ -263,6 +260,11 @@ void VM_Version::initialize() { } } + if (UseAESCTRIntrinsics) { + warning("AES/CTR intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + // GHASH/GCM intrinsics if (has_vis3() && (UseVIS > 2)) { if (FLAG_IS_DEFAULT(UseGHASHIntrinsics)) { @@ -357,6 +359,11 @@ void VM_Version::initialize() { FLAG_SET_DEFAULT(UseCRC32Intrinsics, false); } + if (UseVectorizedMismatchIntrinsic) { + warning("UseVectorizedMismatchIntrinsic specified, but not available on this CPU."); + FLAG_SET_DEFAULT(UseVectorizedMismatchIntrinsic, false); + } + if (FLAG_IS_DEFAULT(ContendedPaddingWidth) && (cache_line_size > ContendedPaddingWidth)) ContendedPaddingWidth = cache_line_size; @@ -403,7 +410,7 @@ void VM_Version::initialize() { } void VM_Version::print_features() { - tty->print_cr("Version:%s", cpu_features()); + tty->print_cr("Version:%s", _features); } int VM_Version::determine_features() { @@ -439,7 +446,7 @@ int VM_Version::determine_features() { return features; } -static int saved_features = 0; +static uint64_t saved_features = 0; void VM_Version::allow_all() { saved_features = _features; diff --git a/hotspot/src/cpu/sparc/vm/vm_version_sparc.hpp b/hotspot/src/cpu/sparc/vm/vm_version_sparc.hpp index db2d77e1bc1..0609fa8b9a0 100644 --- a/hotspot/src/cpu/sparc/vm/vm_version_sparc.hpp +++ b/hotspot/src/cpu/sparc/vm/vm_version_sparc.hpp @@ -30,6 +30,8 @@ class VM_Version: public Abstract_VM_Version { friend class VMStructs; + friend class JVMCIVMStructs; + protected: enum Feature_Flag { v8_instructions = 0, @@ -96,9 +98,6 @@ protected: niagara1_m = generic_v9_m | niagara1_unique_m }; - static int _features; - static const char* _features_str; - static unsigned int _L2_data_cache_line_size; static unsigned int L2_data_cache_line_size() { return _L2_data_cache_line_size; } @@ -174,8 +173,6 @@ public: // On T4 and newer Sparc BIS to the beginning of cache line always zeros it. static bool has_block_zeroing() { return has_blk_init() && is_T4(); } - static const char* cpu_features() { return _features_str; } - // default prefetch block size on sparc static intx prefetch_data_size() { return L2_data_cache_line_size(); } diff --git a/hotspot/src/cpu/x86/vm/assembler_x86.cpp b/hotspot/src/cpu/x86/vm/assembler_x86.cpp index 015a66b7900..9a7fd0069cf 100644 --- a/hotspot/src/cpu/x86/vm/assembler_x86.cpp +++ b/hotspot/src/cpu/x86/vm/assembler_x86.cpp @@ -772,6 +772,7 @@ address Assembler::locate_operand(address inst, WhichOperand which) { case 0x55: // andnps case 0x56: // orps case 0x57: // xorps + case 0x58: // addpd case 0x59: // mulpd case 0x6E: // movd case 0x7E: // movd @@ -2152,33 +2153,64 @@ void Assembler::movddup(XMMRegister dst, XMMRegister src) { emit_int8(0xC0 | encode); } -void Assembler::kmovwl(KRegister dst, Register src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); +void Assembler::kmovbl(KRegister dst, Register src) { + assert(VM_Version::supports_avx512dq(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(dst, knoreg, src, VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x92); emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::kmovbl(Register dst, KRegister src) { + assert(VM_Version::supports_avx512dq(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0x93); + emit_int8((unsigned char)(0xC0 | encode)); +} + +void Assembler::kmovwl(KRegister dst, Register src) { + assert(VM_Version::supports_evex(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0x92); + emit_int8((unsigned char)(0xC0 | encode)); +} + +void Assembler::kmovwl(Register dst, KRegister src) { + assert(VM_Version::supports_evex(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0x93); + emit_int8((unsigned char)(0xC0 | encode)); +} + void Assembler::kmovdl(KRegister dst, Register src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); - VexSimdPrefix pre = !_legacy_mode_bw ? VEX_SIMD_F2 : VEX_SIMD_NONE; + assert(VM_Version::supports_avx512bw(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(dst, knoreg, src, pre, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x92); emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::kmovdl(Register dst, KRegister src) { + assert(VM_Version::supports_avx512bw(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0x93); + emit_int8((unsigned char)(0xC0 | encode)); +} + void Assembler::kmovql(KRegister dst, KRegister src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); + assert(VM_Version::supports_avx512bw(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(dst, knoreg, src, VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x90); emit_int8((unsigned char)(0xC0 | encode)); } void Assembler::kmovql(KRegister dst, Address src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); + assert(VM_Version::supports_avx512bw(), ""); InstructionMark im(this); InstructionAttr attributes(AVX_128bit, /* vex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); vex_prefix(src, 0, dst->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); @@ -2187,7 +2219,7 @@ void Assembler::kmovql(KRegister dst, Address src) { } void Assembler::kmovql(Address dst, KRegister src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); + assert(VM_Version::supports_avx512bw(), ""); InstructionMark im(this); InstructionAttr attributes(AVX_128bit, /* vex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); vex_prefix(dst, 0, src->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); @@ -2196,46 +2228,53 @@ void Assembler::kmovql(Address dst, KRegister src) { } void Assembler::kmovql(KRegister dst, Register src) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); - VexSimdPrefix pre = !_legacy_mode_bw ? VEX_SIMD_F2 : VEX_SIMD_NONE; - InstructionAttr attributes(AVX_128bit, /* rex_w */ !_legacy_mode_bw, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(dst, knoreg, src, pre, VEX_OPCODE_0F, &attributes); + assert(VM_Version::supports_avx512bw(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x92); emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::kmovql(Register dst, KRegister src) { + assert(VM_Version::supports_avx512bw(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); + int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0x93); + emit_int8((unsigned char)(0xC0 | encode)); +} + // This instruction produces ZF or CF flags void Assembler::kortestbl(KRegister src1, KRegister src2) { - NOT_LP64(assert(VM_Version::supports_avx512dq(), "")); + assert(VM_Version::supports_avx512dq(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(src1, knoreg, src2, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(src1->encoding(), 0, src2->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x98); emit_int8((unsigned char)(0xC0 | encode)); } // This instruction produces ZF or CF flags void Assembler::kortestwl(KRegister src1, KRegister src2) { - NOT_LP64(assert(VM_Version::supports_evex(), "")); + assert(VM_Version::supports_evex(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(src1, knoreg, src2, VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(src1->encoding(), 0, src2->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x98); emit_int8((unsigned char)(0xC0 | encode)); } // This instruction produces ZF or CF flags void Assembler::kortestdl(KRegister src1, KRegister src2) { - NOT_LP64(assert(VM_Version::supports_avx512bw(), "")); + assert(VM_Version::supports_avx512bw(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(src1, knoreg, src2, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(src1->encoding(), 0, src2->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x98); emit_int8((unsigned char)(0xC0 | encode)); } // This instruction produces ZF or CF flags void Assembler::kortestql(KRegister src1, KRegister src2) { - NOT_LP64(assert(VM_Version::supports_avx512bw(), "")); + assert(VM_Version::supports_avx512bw(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ true, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = kreg_prefix_and_encode(src1, knoreg, src2, VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); + int encode = vex_prefix_and_encode(src1->encoding(), 0, src2->encoding(), VEX_SIMD_NONE, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0x98); emit_int8((unsigned char)(0xC0 | encode)); } @@ -2375,7 +2414,7 @@ void Assembler::vmovdqu(Address dst, XMMRegister src) { // Move Unaligned EVEX enabled Vector (programmable : 8,16,32,64) void Assembler::evmovdqub(XMMRegister dst, XMMRegister src, int vector_len) { assert(VM_Version::supports_evex(), ""); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x6F); emit_int8((unsigned char)(0xC0 | encode)); @@ -2395,7 +2434,7 @@ void Assembler::evmovdqub(Address dst, XMMRegister src, int vector_len) { assert(VM_Version::supports_evex(), ""); assert(src != xnoreg, "sanity"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); vex_prefix(dst, 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x7F); @@ -2404,7 +2443,7 @@ void Assembler::evmovdqub(Address dst, XMMRegister src, int vector_len) { void Assembler::evmovdquw(XMMRegister dst, XMMRegister src, int vector_len) { assert(VM_Version::supports_evex(), ""); - InstructionAttr attributes(vector_len, /* vex_w */ true, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ true, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x6F); emit_int8((unsigned char)(0xC0 | encode)); @@ -2424,7 +2463,7 @@ void Assembler::evmovdquw(Address dst, XMMRegister src, int vector_len) { assert(VM_Version::supports_evex(), ""); assert(src != xnoreg, "sanity"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ true, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ true, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); vex_prefix(dst, 0, src->encoding(), VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x7F); @@ -3069,7 +3108,7 @@ void Assembler::packuswb(XMMRegister dst, Address src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); assert((UseAVX > 0), "SSE mode requires address alignment 16 bytes"); InstructionMark im(this); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FV, /* input_size_in_bits */ EVEX_32bit); simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x67); @@ -3078,7 +3117,7 @@ void Assembler::packuswb(XMMRegister dst, Address src) { void Assembler::packuswb(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x67); emit_int8((unsigned char)(0xC0 | encode)); @@ -3086,7 +3125,7 @@ void Assembler::packuswb(XMMRegister dst, XMMRegister src) { void Assembler::vpackuswb(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "some form of AVX must be enabled"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x67); @@ -3128,7 +3167,7 @@ void Assembler::pcmpestri(XMMRegister dst, XMMRegister src, int imm8) { // In this context, the dst vector contains the components that are equal, non equal components are zeroed in dst void Assembler::pcmpeqb(XMMRegister dst, XMMRegister src) { - NOT_LP64(assert(VM_Version::supports_sse2(), "")); + assert(VM_Version::supports_sse2(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ false, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x74); @@ -3148,16 +3187,28 @@ void Assembler::vpcmpeqb(XMMRegister dst, XMMRegister nds, XMMRegister src, int // In this context, kdst is written the mask used to process the equal components void Assembler::evpcmpeqb(KRegister kdst, XMMRegister nds, XMMRegister src, int vector_len) { assert(VM_Version::supports_avx512bw(), ""); - InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(kdst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x74); emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::evpcmpeqb(KRegister kdst, XMMRegister nds, Address src, int vector_len) { + assert(VM_Version::supports_avx512bw(), ""); + InstructionMark im(this); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ true, /* uses_vl */ true); + attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); + int nds_enc = nds->is_valid() ? nds->encoding() : 0; + int dst_enc = kdst->encoding(); + vex_prefix(src, nds_enc, dst_enc, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8(0x74); + emit_operand(as_Register(dst_enc), src); +} + // In this context, the dst vector contains the components that are equal, non equal components are zeroed in dst void Assembler::pcmpeqw(XMMRegister dst, XMMRegister src) { - NOT_LP64(assert(VM_Version::supports_sse2(), "")); + assert(VM_Version::supports_sse2(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ false, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x75); @@ -3177,16 +3228,28 @@ void Assembler::vpcmpeqw(XMMRegister dst, XMMRegister nds, XMMRegister src, int // In this context, kdst is written the mask used to process the equal components void Assembler::evpcmpeqw(KRegister kdst, XMMRegister nds, XMMRegister src, int vector_len) { assert(VM_Version::supports_avx512bw(), ""); - InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(kdst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x75); emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::evpcmpeqw(KRegister kdst, XMMRegister nds, Address src, int vector_len) { + assert(VM_Version::supports_avx512bw(), ""); + InstructionMark im(this); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ true, /* uses_vl */ true); + attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); + int nds_enc = nds->is_valid() ? nds->encoding() : 0; + int dst_enc = kdst->encoding(); + vex_prefix(src, nds_enc, dst_enc, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8(0x75); + emit_operand(as_Register(dst_enc), src); +} + // In this context, the dst vector contains the components that are equal, non equal components are zeroed in dst void Assembler::pcmpeqd(XMMRegister dst, XMMRegister src) { - NOT_LP64(assert(VM_Version::supports_sse2(), "")); + assert(VM_Version::supports_sse2(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ false, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x76); @@ -3213,9 +3276,21 @@ void Assembler::evpcmpeqd(KRegister kdst, XMMRegister nds, XMMRegister src, int emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::evpcmpeqd(KRegister kdst, XMMRegister nds, Address src, int vector_len) { + assert(VM_Version::supports_evex(), ""); + InstructionMark im(this); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ false, /* uses_vl */ true); + attributes.set_address_attributes(/* tuple_type */ EVEX_FV, /* input_size_in_bits */ EVEX_32bit); + int nds_enc = nds->is_valid() ? nds->encoding() : 0; + int dst_enc = kdst->encoding(); + vex_prefix(src, nds_enc, dst_enc, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8(0x76); + emit_operand(as_Register(dst_enc), src); +} + // In this context, the dst vector contains the components that are equal, non equal components are zeroed in dst void Assembler::pcmpeqq(XMMRegister dst, XMMRegister src) { - NOT_LP64(assert(VM_Version::supports_sse4_1(), "")); + assert(VM_Version::supports_sse4_1(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ true, /* no_mask_reg */ false, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x29); @@ -3274,21 +3349,41 @@ void Assembler::vpmovmskb(Register dst, XMMRegister src) { void Assembler::pextrd(Register dst, XMMRegister src, int imm8) { assert(VM_Version::supports_sse4_1(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = simd_prefix_and_encode(as_XMMRegister(dst->encoding()), xnoreg, src, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + int encode = simd_prefix_and_encode(src, xnoreg, as_XMMRegister(dst->encoding()), VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); emit_int8(0x16); emit_int8((unsigned char)(0xC0 | encode)); emit_int8(imm8); } +void Assembler::pextrd(Address dst, XMMRegister src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_32bit); + simd_prefix(src, xnoreg, dst, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x16); + emit_operand(src, dst); + emit_int8(imm8); +} + void Assembler::pextrq(Register dst, XMMRegister src, int imm8) { assert(VM_Version::supports_sse4_1(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); - int encode = simd_prefix_and_encode(as_XMMRegister(dst->encoding()), xnoreg, src, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + int encode = simd_prefix_and_encode(src, xnoreg, as_XMMRegister(dst->encoding()), VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); emit_int8(0x16); emit_int8((unsigned char)(0xC0 | encode)); emit_int8(imm8); } +void Assembler::pextrq(Address dst, XMMRegister src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_64bit); + simd_prefix(src, xnoreg, dst, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x16); + emit_operand(src, dst); + emit_int8(imm8); +} + void Assembler::pextrw(Register dst, XMMRegister src, int imm8) { assert(VM_Version::supports_sse2(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); @@ -3298,6 +3393,26 @@ void Assembler::pextrw(Register dst, XMMRegister src, int imm8) { emit_int8(imm8); } +void Assembler::pextrw(Address dst, XMMRegister src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_16bit); + simd_prefix(src, xnoreg, dst, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8((unsigned char)0x15); + emit_operand(src, dst); + emit_int8(imm8); +} + +void Assembler::pextrb(Address dst, XMMRegister src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_8bit); + simd_prefix(src, xnoreg, dst, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x14); + emit_operand(src, dst); + emit_int8(imm8); +} + void Assembler::pinsrd(XMMRegister dst, Register src, int imm8) { assert(VM_Version::supports_sse4_1(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); @@ -3307,6 +3422,16 @@ void Assembler::pinsrd(XMMRegister dst, Register src, int imm8) { emit_int8(imm8); } +void Assembler::pinsrd(XMMRegister dst, Address src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_32bit); + simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x22); + emit_operand(dst,src); + emit_int8(imm8); +} + void Assembler::pinsrq(XMMRegister dst, Register src, int imm8) { assert(VM_Version::supports_sse4_1(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); @@ -3316,6 +3441,16 @@ void Assembler::pinsrq(XMMRegister dst, Register src, int imm8) { emit_int8(imm8); } +void Assembler::pinsrq(XMMRegister dst, Address src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ true, /* legacy_mode */ _legacy_mode_dq, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_64bit); + simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x22); + emit_operand(dst, src); + emit_int8(imm8); +} + void Assembler::pinsrw(XMMRegister dst, Register src, int imm8) { assert(VM_Version::supports_sse2(), ""); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); @@ -3325,10 +3460,30 @@ void Assembler::pinsrw(XMMRegister dst, Register src, int imm8) { emit_int8(imm8); } +void Assembler::pinsrw(XMMRegister dst, Address src, int imm8) { + assert(VM_Version::supports_sse2(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_16bit); + simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8((unsigned char)0xC4); + emit_operand(dst, src); + emit_int8(imm8); +} + +void Assembler::pinsrb(XMMRegister dst, Address src, int imm8) { + assert(VM_Version::supports_sse4_1(), ""); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); + attributes.set_address_attributes(/* tuple_type */ EVEX_T1S, /* input_size_in_bits */ EVEX_8bit); + simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_3A, &attributes); + emit_int8(0x20); + emit_operand(dst, src); + emit_int8(imm8); +} + void Assembler::pmovzxbw(XMMRegister dst, Address src) { assert(VM_Version::supports_sse4_1(), ""); InstructionMark im(this); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); attributes.set_address_attributes(/* tuple_type */ EVEX_HVM, /* input_size_in_bits */ EVEX_NObit); simd_prefix(dst, xnoreg, src, VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x30); @@ -3337,7 +3492,7 @@ void Assembler::pmovzxbw(XMMRegister dst, Address src) { void Assembler::pmovzxbw(XMMRegister dst, XMMRegister src) { assert(VM_Version::supports_sse4_1(), ""); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, xnoreg, src, VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x30); emit_int8((unsigned char)(0xC0 | encode)); @@ -3347,7 +3502,7 @@ void Assembler::vpmovzxbw(XMMRegister dst, Address src, int vector_len) { assert(VM_Version::supports_avx(), ""); InstructionMark im(this); assert(dst != xnoreg, "sanity"); - InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(vector_len, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); attributes.set_address_attributes(/* tuple_type */ EVEX_HVM, /* input_size_in_bits */ EVEX_NObit); vex_prefix(src, 0, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x30); @@ -3452,7 +3607,7 @@ void Assembler::prefix(Prefix p) { void Assembler::pshufb(XMMRegister dst, XMMRegister src) { assert(VM_Version::supports_ssse3(), ""); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x00); emit_int8((unsigned char)(0xC0 | encode)); @@ -3461,7 +3616,7 @@ void Assembler::pshufb(XMMRegister dst, XMMRegister src) { void Assembler::pshufb(XMMRegister dst, Address src) { assert(VM_Version::supports_ssse3(), ""); InstructionMark im(this); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x00); @@ -3495,7 +3650,7 @@ void Assembler::pshufd(XMMRegister dst, Address src, int mode) { void Assembler::pshuflw(XMMRegister dst, XMMRegister src, int mode) { assert(isByte(mode), "invalid value"); NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); int encode = simd_prefix_and_encode(dst, xnoreg, src, VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x70); emit_int8((unsigned char)(0xC0 | encode)); @@ -3507,7 +3662,7 @@ void Assembler::pshuflw(XMMRegister dst, Address src, int mode) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); assert((UseAVX > 0), "SSE mode requires address alignment 16 bytes"); InstructionMark im(this); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ false); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ false); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); simd_prefix(dst, xnoreg, src, VEX_SIMD_F2, VEX_OPCODE_0F, &attributes); emit_int8(0x70); @@ -4112,6 +4267,12 @@ void Assembler::xorl(Register dst, Register src) { emit_arith(0x33, 0xC0, dst, src); } +void Assembler::xorb(Register dst, Address src) { + InstructionMark im(this); + prefix(src, dst); + emit_int8(0x32); + emit_operand(dst, src); +} // AVX 3-operands scalar float-point arithmetic instructions @@ -4287,6 +4448,17 @@ void Assembler::addpd(XMMRegister dst, XMMRegister src) { emit_int8((unsigned char)(0xC0 | encode)); } +void Assembler::addpd(XMMRegister dst, Address src) { + NOT_LP64(assert(VM_Version::supports_sse2(), "")); + InstructionMark im(this); + InstructionAttr attributes(AVX_128bit, /* rex_w */ VM_Version::supports_evex(), /* legacy_mode */ false, /* no_mask_reg */ false, /* uses_vl */ true); + attributes.set_address_attributes(/* tuple_type */ EVEX_FV, /* input_size_in_bits */ EVEX_64bit); + simd_prefix(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); + emit_int8(0x58); + emit_operand(dst, src); +} + + void Assembler::addps(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ false, /* no_mask_reg */ false, /* uses_vl */ true); @@ -4723,7 +4895,7 @@ void Assembler::vphaddd(XMMRegister dst, XMMRegister nds, XMMRegister src, int v void Assembler::paddb(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xFC); emit_int8((unsigned char)(0xC0 | encode)); @@ -4731,7 +4903,7 @@ void Assembler::paddb(XMMRegister dst, XMMRegister src) { void Assembler::paddw(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xFD); emit_int8((unsigned char)(0xC0 | encode)); @@ -4771,7 +4943,7 @@ void Assembler::phaddd(XMMRegister dst, XMMRegister src) { void Assembler::vpaddb(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xFC); @@ -4780,7 +4952,7 @@ void Assembler::vpaddb(XMMRegister dst, XMMRegister nds, XMMRegister src, int ve void Assembler::vpaddw(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xFD); @@ -4808,7 +4980,7 @@ void Assembler::vpaddq(XMMRegister dst, XMMRegister nds, XMMRegister src, int ve void Assembler::vpaddb(XMMRegister dst, XMMRegister nds, Address src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); int nds_enc = nds->is_valid() ? nds->encoding() : 0; vex_prefix(src, nds_enc, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); @@ -4819,7 +4991,7 @@ void Assembler::vpaddb(XMMRegister dst, XMMRegister nds, Address src, int vector void Assembler::vpaddw(XMMRegister dst, XMMRegister nds, Address src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); int nds_enc = nds->is_valid() ? nds->encoding() : 0; vex_prefix(src, nds_enc, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); @@ -4851,7 +5023,7 @@ void Assembler::vpaddq(XMMRegister dst, XMMRegister nds, Address src, int vector void Assembler::psubb(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF8); emit_int8((unsigned char)(0xC0 | encode)); @@ -4859,7 +5031,7 @@ void Assembler::psubb(XMMRegister dst, XMMRegister src) { void Assembler::psubw(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF9); emit_int8((unsigned char)(0xC0 | encode)); @@ -4882,7 +5054,7 @@ void Assembler::psubq(XMMRegister dst, XMMRegister src) { void Assembler::vpsubb(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF8); @@ -4891,7 +5063,7 @@ void Assembler::vpsubb(XMMRegister dst, XMMRegister nds, XMMRegister src, int ve void Assembler::vpsubw(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF9); @@ -4919,7 +5091,7 @@ void Assembler::vpsubq(XMMRegister dst, XMMRegister nds, XMMRegister src, int ve void Assembler::vpsubb(XMMRegister dst, XMMRegister nds, Address src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); int nds_enc = nds->is_valid() ? nds->encoding() : 0; vex_prefix(src, nds_enc, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); @@ -4930,7 +5102,7 @@ void Assembler::vpsubb(XMMRegister dst, XMMRegister nds, Address src, int vector void Assembler::vpsubw(XMMRegister dst, XMMRegister nds, Address src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); int nds_enc = nds->is_valid() ? nds->encoding() : 0; vex_prefix(src, nds_enc, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); @@ -4962,7 +5134,7 @@ void Assembler::vpsubq(XMMRegister dst, XMMRegister nds, Address src, int vector void Assembler::pmullw(XMMRegister dst, XMMRegister src) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, src, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xD5); emit_int8((unsigned char)(0xC0 | encode)); @@ -4978,7 +5150,7 @@ void Assembler::pmulld(XMMRegister dst, XMMRegister src) { void Assembler::vpmullw(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int nds_enc = nds->is_valid() ? nds->encoding() : 0; int encode = vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xD5); @@ -5006,7 +5178,7 @@ void Assembler::vpmullq(XMMRegister dst, XMMRegister nds, XMMRegister src, int v void Assembler::vpmullw(XMMRegister dst, XMMRegister nds, Address src, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); InstructionMark im(this); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); attributes.set_address_attributes(/* tuple_type */ EVEX_FVM, /* input_size_in_bits */ EVEX_NObit); int nds_enc = nds->is_valid() ? nds->encoding() : 0; vex_prefix(src, nds_enc, dst->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); @@ -5039,7 +5211,7 @@ void Assembler::vpmullq(XMMRegister dst, XMMRegister nds, Address src, int vecto // Shift packed integers left by specified number of bits. void Assembler::psllw(XMMRegister dst, int shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM6 is for /6 encoding: 66 0F 71 /6 ib int encode = simd_prefix_and_encode(xmm6, dst, dst, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5069,7 +5241,7 @@ void Assembler::psllq(XMMRegister dst, int shift) { void Assembler::psllw(XMMRegister dst, XMMRegister shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, shift, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5093,7 +5265,7 @@ void Assembler::psllq(XMMRegister dst, XMMRegister shift) { void Assembler::vpsllw(XMMRegister dst, XMMRegister src, int shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM6 is for /6 encoding: 66 0F 71 /6 ib int encode = vex_prefix_and_encode(xmm6->encoding(), dst->encoding(), src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5124,7 +5296,7 @@ void Assembler::vpsllq(XMMRegister dst, XMMRegister src, int shift, int vector_l void Assembler::vpsllw(XMMRegister dst, XMMRegister src, XMMRegister shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), src->encoding(), shift->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xF1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5149,7 +5321,7 @@ void Assembler::vpsllq(XMMRegister dst, XMMRegister src, XMMRegister shift, int // Shift packed integers logically right by specified number of bits. void Assembler::psrlw(XMMRegister dst, int shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM2 is for /2 encoding: 66 0F 71 /2 ib int encode = simd_prefix_and_encode(xmm2, dst, dst, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5181,7 +5353,7 @@ void Assembler::psrlq(XMMRegister dst, int shift) { void Assembler::psrlw(XMMRegister dst, XMMRegister shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, shift, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xD1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5205,7 +5377,7 @@ void Assembler::psrlq(XMMRegister dst, XMMRegister shift) { void Assembler::vpsrlw(XMMRegister dst, XMMRegister src, int shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM2 is for /2 encoding: 66 0F 71 /2 ib int encode = vex_prefix_and_encode(xmm2->encoding(), dst->encoding(), src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5235,7 +5407,7 @@ void Assembler::vpsrlq(XMMRegister dst, XMMRegister src, int shift, int vector_l void Assembler::vpsrlw(XMMRegister dst, XMMRegister src, XMMRegister shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), src->encoding(), shift->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xD1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5260,7 +5432,7 @@ void Assembler::vpsrlq(XMMRegister dst, XMMRegister src, XMMRegister shift, int // Shift packed integers arithmetically right by specified number of bits. void Assembler::psraw(XMMRegister dst, int shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM4 is for /4 encoding: 66 0F 71 /4 ib int encode = simd_prefix_and_encode(xmm4, dst, dst, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5280,7 +5452,7 @@ void Assembler::psrad(XMMRegister dst, int shift) { void Assembler::psraw(XMMRegister dst, XMMRegister shift) { NOT_LP64(assert(VM_Version::supports_sse2(), "")); - InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_128bit, /* rex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = simd_prefix_and_encode(dst, dst, shift, VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xE1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5296,7 +5468,7 @@ void Assembler::psrad(XMMRegister dst, XMMRegister shift) { void Assembler::vpsraw(XMMRegister dst, XMMRegister src, int shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); // XMM4 is for /4 encoding: 66 0F 71 /4 ib int encode = vex_prefix_and_encode(xmm4->encoding(), dst->encoding(), src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8(0x71); @@ -5316,7 +5488,7 @@ void Assembler::vpsrad(XMMRegister dst, XMMRegister src, int shift, int vector_l void Assembler::vpsraw(XMMRegister dst, XMMRegister src, XMMRegister shift, int vector_len) { assert(UseAVX > 0, "requires some form of AVX"); - InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(vector_len, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), src->encoding(), shift->encoding(), VEX_SIMD_66, VEX_OPCODE_0F, &attributes); emit_int8((unsigned char)0xE1); emit_int8((unsigned char)(0xC0 | encode)); @@ -5706,7 +5878,7 @@ void Assembler::vpbroadcastd(XMMRegister dst, XMMRegister src) { // duplicate 2-bytes integer data from src into 16 locations in dest void Assembler::vpbroadcastw(XMMRegister dst, XMMRegister src) { assert(VM_Version::supports_avx2(), ""); - InstructionAttr attributes(AVX_256bit, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ false, /* uses_vl */ true); + InstructionAttr attributes(AVX_256bit, /* vex_w */ false, /* legacy_mode */ _legacy_mode_bw, /* no_mask_reg */ true, /* uses_vl */ true); int encode = vex_prefix_and_encode(dst->encoding(), 0, src->encoding(), VEX_SIMD_66, VEX_OPCODE_0F_38, &attributes); emit_int8(0x79); emit_int8((unsigned char)(0xC0 | encode)); @@ -6573,18 +6745,6 @@ int Assembler::simd_prefix_and_encode(XMMRegister dst, XMMRegister nds, XMMRegis } } -int Assembler::kreg_prefix_and_encode(KRegister dst, KRegister nds, KRegister src, VexSimdPrefix pre, - VexOpcode opc, InstructionAttr *attributes) { - int nds_enc = nds->is_valid() ? nds->encoding() : 0; - return vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), pre, opc, attributes); -} - -int Assembler::kreg_prefix_and_encode(KRegister dst, KRegister nds, Register src, VexSimdPrefix pre, - VexOpcode opc, InstructionAttr *attributes) { - int nds_enc = nds->is_valid() ? nds->encoding() : 0; - return vex_prefix_and_encode(dst->encoding(), nds_enc, src->encoding(), pre, opc, attributes); -} - void Assembler::cmppd(XMMRegister dst, XMMRegister nds, XMMRegister src, int cop, int vector_len) { assert(VM_Version::supports_avx(), ""); assert(!VM_Version::supports_evex(), ""); diff --git a/hotspot/src/cpu/x86/vm/assembler_x86.hpp b/hotspot/src/cpu/x86/vm/assembler_x86.hpp index a5f493905fc..45ecc01f24a 100644 --- a/hotspot/src/cpu/x86/vm/assembler_x86.hpp +++ b/hotspot/src/cpu/x86/vm/assembler_x86.hpp @@ -655,12 +655,6 @@ private: int simd_prefix_and_encode(XMMRegister dst, XMMRegister nds, XMMRegister src, VexSimdPrefix pre, VexOpcode opc, InstructionAttr *attributes); - int kreg_prefix_and_encode(KRegister dst, KRegister nds, KRegister src, VexSimdPrefix pre, - VexOpcode opc, InstructionAttr *attributes); - - int kreg_prefix_and_encode(KRegister dst, KRegister nds, Register src, VexSimdPrefix pre, - VexOpcode opc, InstructionAttr *attributes); - // Helper functions for groups of instructions void emit_arith_b(int op1, int op2, Register dst, int imm8); @@ -1331,12 +1325,17 @@ private: void movddup(XMMRegister dst, XMMRegister src); + void kmovbl(KRegister dst, Register src); + void kmovbl(Register dst, KRegister src); void kmovwl(KRegister dst, Register src); + void kmovwl(Register dst, KRegister src); void kmovdl(KRegister dst, Register src); + void kmovdl(Register dst, KRegister src); void kmovql(KRegister dst, KRegister src); - void kmovql(KRegister dst, Register src); void kmovql(Address dst, KRegister src); void kmovql(KRegister dst, Address src); + void kmovql(KRegister dst, Register src); + void kmovql(Register dst, KRegister src); void kortestbl(KRegister dst, KRegister src); void kortestwl(KRegister dst, KRegister src); @@ -1521,14 +1520,17 @@ private: void pcmpeqb(XMMRegister dst, XMMRegister src); void vpcmpeqb(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); void evpcmpeqb(KRegister kdst, XMMRegister nds, XMMRegister src, int vector_len); + void evpcmpeqb(KRegister kdst, XMMRegister nds, Address src, int vector_len); void pcmpeqw(XMMRegister dst, XMMRegister src); void vpcmpeqw(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); void evpcmpeqw(KRegister kdst, XMMRegister nds, XMMRegister src, int vector_len); + void evpcmpeqw(KRegister kdst, XMMRegister nds, Address src, int vector_len); void pcmpeqd(XMMRegister dst, XMMRegister src); void vpcmpeqd(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); void evpcmpeqd(KRegister kdst, XMMRegister nds, XMMRegister src, int vector_len); + void evpcmpeqd(KRegister kdst, XMMRegister nds, Address src, int vector_len); void pcmpeqq(XMMRegister dst, XMMRegister src); void vpcmpeqq(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); @@ -1541,14 +1543,22 @@ private: // SSE 4.1 extract void pextrd(Register dst, XMMRegister src, int imm8); void pextrq(Register dst, XMMRegister src, int imm8); + void pextrd(Address dst, XMMRegister src, int imm8); + void pextrq(Address dst, XMMRegister src, int imm8); + void pextrb(Address dst, XMMRegister src, int imm8); // SSE 2 extract void pextrw(Register dst, XMMRegister src, int imm8); + void pextrw(Address dst, XMMRegister src, int imm8); // SSE 4.1 insert void pinsrd(XMMRegister dst, Register src, int imm8); void pinsrq(XMMRegister dst, Register src, int imm8); + void pinsrd(XMMRegister dst, Address src, int imm8); + void pinsrq(XMMRegister dst, Address src, int imm8); + void pinsrb(XMMRegister dst, Address src, int imm8); // SSE 2 insert void pinsrw(XMMRegister dst, Register src, int imm8); + void pinsrw(XMMRegister dst, Address src, int imm8); // SSE4.1 packed move void pmovzxbw(XMMRegister dst, XMMRegister src); @@ -1760,6 +1770,8 @@ private: void xorl(Register dst, Address src); void xorl(Register dst, Register src); + void xorb(Register dst, Address src); + void xorq(Register dst, Address src); void xorq(Register dst, Register src); @@ -1789,6 +1801,7 @@ private: // Add Packed Floating-Point Values void addpd(XMMRegister dst, XMMRegister src); + void addpd(XMMRegister dst, Address src); void addps(XMMRegister dst, XMMRegister src); void vaddpd(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); void vaddps(XMMRegister dst, XMMRegister nds, XMMRegister src, int vector_len); diff --git a/hotspot/src/cpu/x86/vm/c1_LIRAssembler_x86.cpp b/hotspot/src/cpu/x86/vm/c1_LIRAssembler_x86.cpp index bfe8b8c86e9..963c1b77df1 100644 --- a/hotspot/src/cpu/x86/vm/c1_LIRAssembler_x86.cpp +++ b/hotspot/src/cpu/x86/vm/c1_LIRAssembler_x86.cpp @@ -2381,9 +2381,6 @@ void LIR_Assembler::intrinsic_op(LIR_Code code, LIR_Opr value, LIR_Opr unused, L // Should consider not saving rbx, if not necessary __ trigfunc('t', op->as_Op2()->fpu_stack_size()); break; - case lir_pow : - __ pow_with_fallback(op->as_Op2()->fpu_stack_size()); - break; default : ShouldNotReachHere(); } } else { diff --git a/hotspot/src/cpu/x86/vm/c1_LIRGenerator_x86.cpp b/hotspot/src/cpu/x86/vm/c1_LIRGenerator_x86.cpp index 8303fe989c8..63073ac4648 100644 --- a/hotspot/src/cpu/x86/vm/c1_LIRGenerator_x86.cpp +++ b/hotspot/src/cpu/x86/vm/c1_LIRGenerator_x86.cpp @@ -736,19 +736,6 @@ void LIRGenerator::do_CompareAndSwap(Intrinsic* x, ValueType* type) { obj.load_item(); offset.load_nonconstant(); - if (type == objectType) { - cmp.load_item_force(FrameMap::rax_oop_opr); - val.load_item(); - } else if (type == intType) { - cmp.load_item_force(FrameMap::rax_opr); - val.load_item(); - } else if (type == longType) { - cmp.load_item_force(FrameMap::long0_opr); - val.load_item_force(FrameMap::long1_opr); - } else { - ShouldNotReachHere(); - } - LIR_Opr addr = new_pointer_register(); LIR_Address* a; if(offset.result()->is_constant()) { @@ -785,6 +772,19 @@ void LIRGenerator::do_CompareAndSwap(Intrinsic* x, ValueType* type) { true /* do_load */, false /* patch */, NULL); } + if (type == objectType) { + cmp.load_item_force(FrameMap::rax_oop_opr); + val.load_item(); + } else if (type == intType) { + cmp.load_item_force(FrameMap::rax_opr); + val.load_item(); + } else if (type == longType) { + cmp.load_item_force(FrameMap::long0_opr); + val.load_item_force(FrameMap::long1_opr); + } else { + ShouldNotReachHere(); + } + LIR_Opr ill = LIR_OprFact::illegalOpr; // for convenience if (type == objectType) __ cas_obj(addr, cmp.result(), val.result(), ill, ill); @@ -810,7 +810,8 @@ void LIRGenerator::do_CompareAndSwap(Intrinsic* x, ValueType* type) { void LIRGenerator::do_MathIntrinsic(Intrinsic* x) { assert(x->number_of_arguments() == 1 || (x->number_of_arguments() == 2 && x->id() == vmIntrinsics::_dpow), "wrong type"); - if (x->id() == vmIntrinsics::_dexp || x->id() == vmIntrinsics::_dlog) { + if (x->id() == vmIntrinsics::_dexp || x->id() == vmIntrinsics::_dlog || + x->id() == vmIntrinsics::_dpow) { do_LibmIntrinsic(x); return; } @@ -824,7 +825,6 @@ void LIRGenerator::do_MathIntrinsic(Intrinsic* x) { case vmIntrinsics::_dcos: case vmIntrinsics::_dtan: case vmIntrinsics::_dlog10: - case vmIntrinsics::_dpow: use_fpu = true; } } else { @@ -874,7 +874,6 @@ void LIRGenerator::do_MathIntrinsic(Intrinsic* x) { case vmIntrinsics::_dcos: __ cos (calc_input, calc_result, tmp1, tmp2); break; case vmIntrinsics::_dtan: __ tan (calc_input, calc_result, tmp1, tmp2); break; case vmIntrinsics::_dlog10: __ log10(calc_input, calc_result, tmp1); break; - case vmIntrinsics::_dpow: __ pow (calc_input, calc_input2, calc_result, tmp1, tmp2, FrameMap::rax_opr, FrameMap::rcx_opr, FrameMap::rdx_opr); break; default: ShouldNotReachHere(); } @@ -890,11 +889,25 @@ void LIRGenerator::do_LibmIntrinsic(Intrinsic* x) { LIR_Opr calc_result = rlock_result(x); LIR_Opr result_reg = result_register_for(x->type()); - BasicTypeList signature(1); - signature.append(T_DOUBLE); - CallingConvention* cc = frame_map()->c_calling_convention(&signature); + CallingConvention* cc = NULL; - value.load_item_force(cc->at(0)); + if (x->id() == vmIntrinsics::_dpow) { + LIRItem value1(x->argument_at(1), this); + + value1.set_destroys_register(); + + BasicTypeList signature(2); + signature.append(T_DOUBLE); + signature.append(T_DOUBLE); + cc = frame_map()->c_calling_convention(&signature); + value.load_item_force(cc->at(0)); + value1.load_item_force(cc->at(1)); + } else { + BasicTypeList signature(1); + signature.append(T_DOUBLE); + cc = frame_map()->c_calling_convention(&signature); + value.load_item_force(cc->at(0)); + } #ifndef _LP64 LIR_Opr tmp = FrameMap::fpu0_double_opr; @@ -915,6 +928,14 @@ void LIRGenerator::do_LibmIntrinsic(Intrinsic* x) { __ call_runtime_leaf(CAST_FROM_FN_PTR(address, SharedRuntime::dlog), getThreadTemp(), result_reg, cc->args()); } break; + case vmIntrinsics::_dpow: + if (VM_Version::supports_sse2()) { + __ call_runtime_leaf(StubRoutines::dpow(), getThreadTemp(), result_reg, cc->args()); + } + else { + __ call_runtime_leaf(CAST_FROM_FN_PTR(address, SharedRuntime::dpow), getThreadTemp(), result_reg, cc->args()); + } + break; default: ShouldNotReachHere(); } #else @@ -925,6 +946,9 @@ void LIRGenerator::do_LibmIntrinsic(Intrinsic* x) { case vmIntrinsics::_dlog: __ call_runtime_leaf(StubRoutines::dlog(), getThreadTemp(), result_reg, cc->args()); break; + case vmIntrinsics::_dpow: + __ call_runtime_leaf(StubRoutines::dpow(), getThreadTemp(), result_reg, cc->args()); + break; } #endif __ move(result_reg, calc_result); diff --git a/hotspot/src/cpu/x86/vm/c1_LinearScan_x86.cpp b/hotspot/src/cpu/x86/vm/c1_LinearScan_x86.cpp index 462aa855c5f..52924b6e8dc 100644 --- a/hotspot/src/cpu/x86/vm/c1_LinearScan_x86.cpp +++ b/hotspot/src/cpu/x86/vm/c1_LinearScan_x86.cpp @@ -840,53 +840,6 @@ void FpuStackAllocator::handle_op2(LIR_Op2* op2) { break; } - case lir_pow: { - // pow needs two temporary fpu stack slots, so there are two temporary - // registers (stored in tmp1 and tmp2 of the operation). - // the stack allocator must guarantee that the stack slots are really free, - // otherwise there might be a stack overflow. - assert(left->is_fpu_register(), "must be"); - assert(right->is_fpu_register(), "must be"); - assert(res->is_fpu_register(), "must be"); - - assert(op2->tmp1_opr()->is_fpu_register(), "tmp1 is the first temporary register"); - assert(op2->tmp2_opr()->is_fpu_register(), "tmp2 is the second temporary register"); - assert(fpu_num(left) != fpu_num(right) && fpu_num(left) != fpu_num(op2->tmp1_opr()) && fpu_num(left) != fpu_num(op2->tmp2_opr()) && fpu_num(left) != fpu_num(res), "need distinct temp registers"); - assert(fpu_num(right) != fpu_num(op2->tmp1_opr()) && fpu_num(right) != fpu_num(op2->tmp2_opr()) && fpu_num(right) != fpu_num(res), "need distinct temp registers"); - assert(fpu_num(op2->tmp1_opr()) != fpu_num(op2->tmp2_opr()) && fpu_num(op2->tmp1_opr()) != fpu_num(res), "need distinct temp registers"); - assert(fpu_num(op2->tmp2_opr()) != fpu_num(res), "need distinct temp registers"); - - insert_free_if_dead(op2->tmp1_opr()); - insert_free_if_dead(op2->tmp2_opr()); - - // Must bring both operands to top of stack with following operand ordering: - // * fpu stack before pow: ... right left - // * fpu stack after pow: ... left - - insert_free_if_dead(res, right); - - if (tos_offset(right) != 1) { - insert_exchange(right); - insert_exchange(1); - } - insert_exchange(left); - assert(tos_offset(right) == 1, "check"); - assert(tos_offset(left) == 0, "check"); - - new_left = to_fpu_stack_top(left); - new_right = to_fpu_stack(right); - - op2->set_fpu_stack_size(sim()->stack_size()); - assert(sim()->stack_size() <= 6, "at least two stack slots must be free"); - - sim()->pop(); - - do_rename(right, res); - - new_res = to_fpu_stack_top(res); - break; - } - default: { assert(false, "missed a fpu-operation"); } diff --git a/hotspot/src/cpu/x86/vm/globalDefinitions_x86.hpp b/hotspot/src/cpu/x86/vm/globalDefinitions_x86.hpp index 5118e01130b..40f3d2fd69c 100644 --- a/hotspot/src/cpu/x86/vm/globalDefinitions_x86.hpp +++ b/hotspot/src/cpu/x86/vm/globalDefinitions_x86.hpp @@ -27,6 +27,10 @@ const int StackAlignmentInBytes = 16; +// Indicates whether the C calling conventions require that +// 32-bit integer argument values are extended to 64 bits. +const bool CCallingConventionRequiresIntsAsLongs = false; + #define SUPPORTS_NATIVE_CX8 // The expected size in bytes of a cache line, used to pad data structures. diff --git a/hotspot/src/cpu/x86/vm/globals_x86.hpp b/hotspot/src/cpu/x86/vm/globals_x86.hpp index 07fc7d511b4..57dc051ab5f 100644 --- a/hotspot/src/cpu/x86/vm/globals_x86.hpp +++ b/hotspot/src/cpu/x86/vm/globals_x86.hpp @@ -178,9 +178,6 @@ define_pd_global(bool, PreserveFramePointer, false); "Use RTM Xend instead of Xabort when lock busy") \ \ /* assembler */ \ - product(bool, Use486InstrsOnly, false, \ - "Use 80486 Compliant instruction subset") \ - \ product(bool, UseCountLeadingZerosInstruction, false, \ "Use count leading zeros instruction") \ \ diff --git a/hotspot/src/cpu/x86/vm/interpreter_x86_32.cpp b/hotspot/src/cpu/x86/vm/interpreter_x86_32.cpp index efc8525c333..872abcc2b99 100644 --- a/hotspot/src/cpu/x86/vm/interpreter_x86_32.cpp +++ b/hotspot/src/cpu/x86/vm/interpreter_x86_32.cpp @@ -149,10 +149,15 @@ address TemplateInterpreterGenerator::generate_math_entry(AbstractInterpreter::M break; case Interpreter::java_lang_math_pow: __ fld_d(Address(rsp, 3*wordSize)); // second argument - __ pow_with_fallback(0); - // Store to stack to convert 80bit precision back to 64bits - __ push_fTOS(); - __ pop_fTOS(); + __ subptr(rsp, 4 * wordSize); + __ fstp_d(Address(rsp, 0)); + __ fstp_d(Address(rsp, 2 * wordSize)); + if (VM_Version::supports_sse2()) { + __ call(RuntimeAddress(CAST_FROM_FN_PTR(address, StubRoutines::dpow()))); + } else { + __ call(RuntimeAddress(CAST_FROM_FN_PTR(address, SharedRuntime::dpow))); + } + __ addptr(rsp, 4 * wordSize); break; case Interpreter::java_lang_math_exp: __ subptr(rsp, 2*wordSize); diff --git a/hotspot/src/cpu/x86/vm/interpreter_x86_64.cpp b/hotspot/src/cpu/x86/vm/interpreter_x86_64.cpp index 9e597042095..ed2a8c13240 100644 --- a/hotspot/src/cpu/x86/vm/interpreter_x86_64.cpp +++ b/hotspot/src/cpu/x86/vm/interpreter_x86_64.cpp @@ -255,6 +255,10 @@ address TemplateInterpreterGenerator::generate_math_entry(AbstractInterpreter::M } else if (kind == Interpreter::java_lang_math_log) { __ movdbl(xmm0, Address(rsp, wordSize)); __ call(RuntimeAddress(CAST_FROM_FN_PTR(address, StubRoutines::dlog()))); + } else if (kind == Interpreter::java_lang_math_pow) { + __ movdbl(xmm1, Address(rsp, wordSize)); + __ movdbl(xmm0, Address(rsp, 3 * wordSize)); + __ call(RuntimeAddress(CAST_FROM_FN_PTR(address, StubRoutines::dpow()))); } else { __ fld_d(Address(rsp, wordSize)); switch (kind) { @@ -273,11 +277,6 @@ address TemplateInterpreterGenerator::generate_math_entry(AbstractInterpreter::M case Interpreter::java_lang_math_log10: __ flog10(); break; - case Interpreter::java_lang_math_pow: - __ fld_d(Address(rsp, 3*wordSize)); // second argument (one - // empty stack slot) - __ pow_with_fallback(0); - break; default : ShouldNotReachHere(); } diff --git a/hotspot/src/cpu/x86/vm/macroAssembler_x86.cpp b/hotspot/src/cpu/x86/vm/macroAssembler_x86.cpp index 0bde2c6a516..4381e8aa849 100644 --- a/hotspot/src/cpu/x86/vm/macroAssembler_x86.cpp +++ b/hotspot/src/cpu/x86/vm/macroAssembler_x86.cpp @@ -2277,8 +2277,8 @@ void MacroAssembler::call(AddressLiteral entry) { } } -void MacroAssembler::ic_call(address entry) { - RelocationHolder rh = virtual_call_Relocation::spec(pc()); +void MacroAssembler::ic_call(address entry, jint method_index) { + RelocationHolder rh = virtual_call_Relocation::spec(pc(), method_index); movptr(rax, (intptr_t)Universe::non_oop_word()); call(AddressLiteral(entry, rh)); } @@ -3058,50 +3058,6 @@ void MacroAssembler::mulpd(XMMRegister dst, AddressLiteral src) { } } -void MacroAssembler::pow_exp_core_encoding() { - // kills rax, rcx, rdx - subptr(rsp,sizeof(jdouble)); - // computes 2^X. Stack: X ... - // f2xm1 computes 2^X-1 but only operates on -1<=X<=1. Get int(X) and - // keep it on the thread's stack to compute 2^int(X) later - // then compute 2^(X-int(X)) as (2^(X-int(X)-1+1) - // final result is obtained with: 2^X = 2^int(X) * 2^(X-int(X)) - fld_s(0); // Stack: X X ... - frndint(); // Stack: int(X) X ... - fsuba(1); // Stack: int(X) X-int(X) ... - fistp_s(Address(rsp,0)); // move int(X) as integer to thread's stack. Stack: X-int(X) ... - f2xm1(); // Stack: 2^(X-int(X))-1 ... - fld1(); // Stack: 1 2^(X-int(X))-1 ... - faddp(1); // Stack: 2^(X-int(X)) - // computes 2^(int(X)): add exponent bias (1023) to int(X), then - // shift int(X)+1023 to exponent position. - // Exponent is limited to 11 bits if int(X)+1023 does not fit in 11 - // bits, set result to NaN. 0x000 and 0x7FF are reserved exponent - // values so detect them and set result to NaN. - movl(rax,Address(rsp,0)); - movl(rcx, -2048); // 11 bit mask and valid NaN binary encoding - addl(rax, 1023); - movl(rdx,rax); - shll(rax,20); - // Check that 0 < int(X)+1023 < 2047. Otherwise set rax to NaN. - addl(rdx,1); - // Check that 1 < int(X)+1023+1 < 2048 - // in 3 steps: - // 1- (int(X)+1023+1)&-2048 == 0 => 0 <= int(X)+1023+1 < 2048 - // 2- (int(X)+1023+1)&-2048 != 0 - // 3- (int(X)+1023+1)&-2048 != 1 - // Do 2- first because addl just updated the flags. - cmov32(Assembler::equal,rax,rcx); - cmpl(rdx,1); - cmov32(Assembler::equal,rax,rcx); - testl(rdx,rcx); - cmov32(Assembler::notEqual,rax,rcx); - movl(Address(rsp,4),rax); - movl(Address(rsp,0),0); - fmul_d(Address(rsp,0)); // Stack: 2^X ... - addptr(rsp,sizeof(jdouble)); -} - void MacroAssembler::increase_precision() { subptr(rsp, BytesPerWord); fnstcw(Address(rsp, 0)); @@ -3117,194 +3073,6 @@ void MacroAssembler::restore_precision() { addptr(rsp, BytesPerWord); } -void MacroAssembler::fast_pow() { - // computes X^Y = 2^(Y * log2(X)) - // if fast computation is not possible, result is NaN. Requires - // fallback from user of this macro. - // increase precision for intermediate steps of the computation - BLOCK_COMMENT("fast_pow {"); - increase_precision(); - fyl2x(); // Stack: (Y*log2(X)) ... - pow_exp_core_encoding(); // Stack: exp(X) ... - restore_precision(); - BLOCK_COMMENT("} fast_pow"); -} - -void MacroAssembler::pow_or_exp(int num_fpu_regs_in_use) { - // kills rax, rcx, rdx - // pow and exp needs 2 extra registers on the fpu stack. - Label slow_case, done; - Register tmp = noreg; - if (!VM_Version::supports_cmov()) { - // fcmp needs a temporary so preserve rdx, - tmp = rdx; - } - Register tmp2 = rax; - Register tmp3 = rcx; - - // Stack: X Y - Label x_negative, y_not_2; - - static double two = 2.0; - ExternalAddress two_addr((address)&two); - - // constant maybe too far on 64 bit - lea(tmp2, two_addr); - fld_d(Address(tmp2, 0)); // Stack: 2 X Y - fcmp(tmp, 2, true, false); // Stack: X Y - jcc(Assembler::parity, y_not_2); - jcc(Assembler::notEqual, y_not_2); - - fxch(); fpop(); // Stack: X - fmul(0); // Stack: X*X - - jmp(done); - - bind(y_not_2); - - fldz(); // Stack: 0 X Y - fcmp(tmp, 1, true, false); // Stack: X Y - jcc(Assembler::above, x_negative); - - // X >= 0 - - fld_s(1); // duplicate arguments for runtime call. Stack: Y X Y - fld_s(1); // Stack: X Y X Y - fast_pow(); // Stack: X^Y X Y - fcmp(tmp, 0, false, false); // Stack: X^Y X Y - // X^Y not equal to itself: X^Y is NaN go to slow case. - jcc(Assembler::parity, slow_case); - // get rid of duplicate arguments. Stack: X^Y - if (num_fpu_regs_in_use > 0) { - fxch(); fpop(); - fxch(); fpop(); - } else { - ffree(2); - ffree(1); - } - jmp(done); - - // X <= 0 - bind(x_negative); - - fld_s(1); // Stack: Y X Y - frndint(); // Stack: int(Y) X Y - fcmp(tmp, 2, false, false); // Stack: int(Y) X Y - jcc(Assembler::notEqual, slow_case); - - subptr(rsp, 8); - - // For X^Y, when X < 0, Y has to be an integer and the final - // result depends on whether it's odd or even. We just checked - // that int(Y) == Y. We move int(Y) to gp registers as a 64 bit - // integer to test its parity. If int(Y) is huge and doesn't fit - // in the 64 bit integer range, the integer indefinite value will - // end up in the gp registers. Huge numbers are all even, the - // integer indefinite number is even so it's fine. - -#ifdef ASSERT - // Let's check we don't end up with an integer indefinite number - // when not expected. First test for huge numbers: check whether - // int(Y)+1 == int(Y) which is true for very large numbers and - // those are all even. A 64 bit integer is guaranteed to not - // overflow for numbers where y+1 != y (when precision is set to - // double precision). - Label y_not_huge; - - fld1(); // Stack: 1 int(Y) X Y - fadd(1); // Stack: 1+int(Y) int(Y) X Y - -#ifdef _LP64 - // trip to memory to force the precision down from double extended - // precision - fstp_d(Address(rsp, 0)); - fld_d(Address(rsp, 0)); -#endif - - fcmp(tmp, 1, true, false); // Stack: int(Y) X Y -#endif - - // move int(Y) as 64 bit integer to thread's stack - fistp_d(Address(rsp,0)); // Stack: X Y - -#ifdef ASSERT - jcc(Assembler::notEqual, y_not_huge); - - // Y is huge so we know it's even. It may not fit in a 64 bit - // integer and we don't want the debug code below to see the - // integer indefinite value so overwrite int(Y) on the thread's - // stack with 0. - movl(Address(rsp, 0), 0); - movl(Address(rsp, 4), 0); - - bind(y_not_huge); -#endif - - fld_s(1); // duplicate arguments for runtime call. Stack: Y X Y - fld_s(1); // Stack: X Y X Y - fabs(); // Stack: abs(X) Y X Y - fast_pow(); // Stack: abs(X)^Y X Y - fcmp(tmp, 0, false, false); // Stack: abs(X)^Y X Y - // abs(X)^Y not equal to itself: abs(X)^Y is NaN go to slow case. - - pop(tmp2); - NOT_LP64(pop(tmp3)); - jcc(Assembler::parity, slow_case); - -#ifdef ASSERT - // Check that int(Y) is not integer indefinite value (int - // overflow). Shouldn't happen because for values that would - // overflow, 1+int(Y)==Y which was tested earlier. -#ifndef _LP64 - { - Label integer; - testl(tmp2, tmp2); - jcc(Assembler::notZero, integer); - cmpl(tmp3, 0x80000000); - jcc(Assembler::notZero, integer); - STOP("integer indefinite value shouldn't be seen here"); - bind(integer); - } -#else - { - Label integer; - mov(tmp3, tmp2); // preserve tmp2 for parity check below - shlq(tmp3, 1); - jcc(Assembler::carryClear, integer); - jcc(Assembler::notZero, integer); - STOP("integer indefinite value shouldn't be seen here"); - bind(integer); - } -#endif -#endif - - // get rid of duplicate arguments. Stack: X^Y - if (num_fpu_regs_in_use > 0) { - fxch(); fpop(); - fxch(); fpop(); - } else { - ffree(2); - ffree(1); - } - - testl(tmp2, 1); - jcc(Assembler::zero, done); // X <= 0, Y even: X^Y = abs(X)^Y - // X <= 0, Y even: X^Y = -abs(X)^Y - - fchs(); // Stack: -abs(X)^Y Y - jmp(done); - - // slow case: runtime call - bind(slow_case); - - fpop(); // pop incorrect result or int(Y) - - fp_runtime_fallback(CAST_FROM_FN_PTR(address, SharedRuntime::dpow), 2, num_fpu_regs_in_use); - - // Come here with result in F-TOS - bind(done); -} - void MacroAssembler::fpop() { ffree(); fincstp(); @@ -8014,9 +7782,15 @@ void MacroAssembler::string_compare(Register str1, Register str2, XMMRegister vec1, int ae) { ShortBranchVerifier sbv(this); Label LENGTH_DIFF_LABEL, POP_LABEL, DONE_LABEL, WHILE_HEAD_LABEL; + Label COMPARE_WIDE_VECTORS_LOOP_FAILED; // used only _LP64 && AVX3 int stride, stride2, adr_stride, adr_stride1, adr_stride2; + int stride2x2 = 0x40; Address::ScaleFactor scale, scale1, scale2; + if (ae != StrIntrinsicNode::LL) { + stride2x2 = 0x20; + } + if (ae == StrIntrinsicNode::LU || ae == StrIntrinsicNode::UL) { shrl(cnt2, 1); } @@ -8026,15 +7800,15 @@ void MacroAssembler::string_compare(Register str1, Register str2, movl(result, cnt1); subl(cnt1, cnt2); push(cnt1); - cmov32(Assembler::lessEqual, cnt2, result); + cmov32(Assembler::lessEqual, cnt2, result); // cnt2 = min(cnt1, cnt2) // Is the minimum length zero? testl(cnt2, cnt2); jcc(Assembler::zero, LENGTH_DIFF_LABEL); if (ae == StrIntrinsicNode::LL) { // Load first bytes - load_unsigned_byte(result, Address(str1, 0)); - load_unsigned_byte(cnt1, Address(str2, 0)); + load_unsigned_byte(result, Address(str1, 0)); // result = str1[0] + load_unsigned_byte(cnt1, Address(str2, 0)); // cnt1 = str2[0] } else if (ae == StrIntrinsicNode::UU) { // Load first characters load_unsigned_short(result, Address(str1, 0)); @@ -8075,7 +7849,10 @@ void MacroAssembler::string_compare(Register str1, Register str2, assert(UseSSE >= 4, "SSE4 must be enabled for SSE4.2 intrinsics to be available"); Label COMPARE_WIDE_VECTORS, VECTOR_NOT_EQUAL, COMPARE_WIDE_TAIL, COMPARE_SMALL_STR; Label COMPARE_WIDE_VECTORS_LOOP, COMPARE_16_CHARS, COMPARE_INDEX_CHAR; + Label COMPARE_WIDE_VECTORS_LOOP_AVX2; Label COMPARE_TAIL_LONG; + Label COMPARE_WIDE_VECTORS_LOOP_AVX3; // used only _LP64 && AVX3 + int pcmpmask = 0x19; if (ae == StrIntrinsicNode::LL) { pcmpmask &= ~0x01; @@ -8138,11 +7915,40 @@ void MacroAssembler::string_compare(Register str1, Register str2, } subl(result, stride2); subl(cnt2, stride2); - jccb(Assembler::zero, COMPARE_WIDE_TAIL); + jcc(Assembler::zero, COMPARE_WIDE_TAIL); negptr(result); // In a loop, compare 16-chars (32-bytes) at once using (vpxor+vptest) bind(COMPARE_WIDE_VECTORS_LOOP); + +#ifdef _LP64 + if (VM_Version::supports_avx512vlbw()) { // trying 64 bytes fast loop + cmpl(cnt2, stride2x2); + jccb(Assembler::below, COMPARE_WIDE_VECTORS_LOOP_AVX2); + testl(cnt2, stride2x2-1); // cnt2 holds the vector count + jccb(Assembler::notZero, COMPARE_WIDE_VECTORS_LOOP_AVX2); // means we cannot subtract by 0x40 + + bind(COMPARE_WIDE_VECTORS_LOOP_AVX3); // the hottest loop + if (ae == StrIntrinsicNode::LL || ae == StrIntrinsicNode::UU) { + evmovdquq(vec1, Address(str1, result, scale), Assembler::AVX_512bit); + evpcmpeqb(k7, vec1, Address(str2, result, scale), Assembler::AVX_512bit); // k7 == 11..11, if operands equal, otherwise k7 has some 0 + } else { + vpmovzxbw(vec1, Address(str1, result, scale1), Assembler::AVX_512bit); + evpcmpeqb(k7, vec1, Address(str2, result, scale2), Assembler::AVX_512bit); // k7 == 11..11, if operands equal, otherwise k7 has some 0 + } + kortestql(k7, k7); + jcc(Assembler::aboveEqual, COMPARE_WIDE_VECTORS_LOOP_FAILED); // miscompare + addptr(result, stride2x2); // update since we already compared at this addr + subl(cnt2, stride2x2); // and sub the size too + jccb(Assembler::notZero, COMPARE_WIDE_VECTORS_LOOP_AVX3); + + vpxor(vec1, vec1); + jmpb(COMPARE_WIDE_TAIL); + }//if (VM_Version::supports_avx512vlbw()) +#endif // _LP64 + + + bind(COMPARE_WIDE_VECTORS_LOOP_AVX2); if (ae == StrIntrinsicNode::LL || ae == StrIntrinsicNode::UU) { vmovdqu(vec1, Address(str1, result, scale)); vpxor(vec1, Address(str2, result, scale)); @@ -8151,7 +7957,7 @@ void MacroAssembler::string_compare(Register str1, Register str2, vpxor(vec1, Address(str2, result, scale2)); } vptest(vec1, vec1); - jccb(Assembler::notZero, VECTOR_NOT_EQUAL); + jcc(Assembler::notZero, VECTOR_NOT_EQUAL); addptr(result, stride2); subl(cnt2, stride2); jccb(Assembler::notZero, COMPARE_WIDE_VECTORS_LOOP); @@ -8166,7 +7972,7 @@ void MacroAssembler::string_compare(Register str1, Register str2, movl(result, stride2); movl(cnt2, result); negptr(result); - jmpb(COMPARE_WIDE_VECTORS_LOOP); + jmp(COMPARE_WIDE_VECTORS_LOOP_AVX2); // Identifies the mismatching (higher or lower)16-bytes in the 32-byte vectors. bind(VECTOR_NOT_EQUAL); @@ -8310,6 +8116,34 @@ void MacroAssembler::string_compare(Register str1, Register str2, } jmpb(DONE_LABEL); +#ifdef _LP64 + if (VM_Version::supports_avx512vlbw()) { + + bind(COMPARE_WIDE_VECTORS_LOOP_FAILED); + + kmovql(cnt1, k7); + notq(cnt1); + bsfq(cnt2, cnt1); + if (ae != StrIntrinsicNode::LL) { + // Divide diff by 2 to get number of chars + sarl(cnt2, 1); + } + addq(result, cnt2); + if (ae == StrIntrinsicNode::LL) { + load_unsigned_byte(cnt1, Address(str2, result)); + load_unsigned_byte(result, Address(str1, result)); + } else if (ae == StrIntrinsicNode::UU) { + load_unsigned_short(cnt1, Address(str2, result, scale)); + load_unsigned_short(result, Address(str1, result, scale)); + } else { + load_unsigned_short(cnt1, Address(str2, result, scale2)); + load_unsigned_byte(result, Address(str1, result, scale1)); + } + subl(result, cnt1); + jmpb(POP_LABEL); + }//if (VM_Version::supports_avx512vlbw()) +#endif // _LP64 + // Discard the stored length difference bind(POP_LABEL); pop(cnt1); @@ -8319,6 +8153,7 @@ void MacroAssembler::string_compare(Register str1, Register str2, if(ae == StrIntrinsicNode::UL) { negl(result); } + } // Search for Non-ASCII character (Negative byte value) in a byte array, @@ -8510,13 +8345,53 @@ void MacroAssembler::arrays_equals(bool is_array_equ, Register ary1, Register ar // Compare 32-byte vectors andl(result, 0x0000001f); // tail count (in bytes) andl(limit, 0xffffffe0); // vector count (in bytes) - jccb(Assembler::zero, COMPARE_TAIL); + jcc(Assembler::zero, COMPARE_TAIL); lea(ary1, Address(ary1, limit, Address::times_1)); lea(ary2, Address(ary2, limit, Address::times_1)); negptr(limit); bind(COMPARE_WIDE_VECTORS); + +#ifdef _LP64 + if (VM_Version::supports_avx512vlbw()) { // trying 64 bytes fast loop + Label COMPARE_WIDE_VECTORS_LOOP_AVX2, COMPARE_WIDE_VECTORS_LOOP_AVX3; + + cmpl(limit, -64); + jccb(Assembler::greater, COMPARE_WIDE_VECTORS_LOOP_AVX2); + + bind(COMPARE_WIDE_VECTORS_LOOP_AVX3); // the hottest loop + + evmovdquq(vec1, Address(ary1, limit, Address::times_1), Assembler::AVX_512bit); + evpcmpeqb(k7, vec1, Address(ary2, limit, Address::times_1), Assembler::AVX_512bit); + kortestql(k7, k7); + jcc(Assembler::aboveEqual, FALSE_LABEL); // miscompare + addptr(limit, 64); // update since we already compared at this addr + cmpl(limit, -64); + jccb(Assembler::lessEqual, COMPARE_WIDE_VECTORS_LOOP_AVX3); + + // At this point we may still need to compare -limit+result bytes. + // We could execute the next two instruction and just continue via non-wide path: + // cmpl(limit, 0); + // jcc(Assembler::equal, COMPARE_TAIL); // true + // But since we stopped at the points ary{1,2}+limit which are + // not farther than 64 bytes from the ends of arrays ary{1,2}+result + // (|limit| <= 32 and result < 32), + // we may just compare the last 64 bytes. + // + addptr(result, -64); // it is safe, bc we just came from this area + evmovdquq(vec1, Address(ary1, result, Address::times_1), Assembler::AVX_512bit); + evpcmpeqb(k7, vec1, Address(ary2, result, Address::times_1), Assembler::AVX_512bit); + kortestql(k7, k7); + jcc(Assembler::aboveEqual, FALSE_LABEL); // miscompare + + jmp(TRUE_LABEL); + + bind(COMPARE_WIDE_VECTORS_LOOP_AVX2); + + }//if (VM_Version::supports_avx512vlbw()) +#endif //_LP64 + vmovdqu(vec1, Address(ary1, limit, Address::times_1)); vmovdqu(vec2, Address(ary2, limit, Address::times_1)); vpxor(vec1, vec2); @@ -9454,13 +9329,184 @@ void MacroAssembler::multiply_to_len(Register x, Register xlen, Register y, Regi pop(tmp1); } +void MacroAssembler::vectorized_mismatch(Register obja, Register objb, Register length, Register log2_array_indxscale, + Register result, Register tmp1, Register tmp2, XMMRegister rymm0, XMMRegister rymm1, XMMRegister rymm2){ + assert(UseSSE42Intrinsics, "SSE4.2 must be enabled."); + Label VECTOR32_LOOP, VECTOR16_LOOP, VECTOR8_LOOP, VECTOR4_LOOP; + Label VECTOR16_TAIL, VECTOR8_TAIL, VECTOR4_TAIL; + Label VECTOR32_NOT_EQUAL, VECTOR16_NOT_EQUAL, VECTOR8_NOT_EQUAL, VECTOR4_NOT_EQUAL; + Label SAME_TILL_END, DONE; + Label BYTES_LOOP, BYTES_TAIL, BYTES_NOT_EQUAL; + + //scale is in rcx in both Win64 and Unix + ShortBranchVerifier sbv(this); + + shlq(length); + xorq(result, result); + + cmpq(length, 8); + jcc(Assembler::equal, VECTOR8_LOOP); + jcc(Assembler::less, VECTOR4_TAIL); + + if (UseAVX >= 2){ + + cmpq(length, 16); + jcc(Assembler::equal, VECTOR16_LOOP); + jcc(Assembler::less, VECTOR8_LOOP); + + cmpq(length, 32); + jccb(Assembler::less, VECTOR16_TAIL); + + subq(length, 32); + bind(VECTOR32_LOOP); + vmovdqu(rymm0, Address(obja, result)); + vmovdqu(rymm1, Address(objb, result)); + vpxor(rymm2, rymm0, rymm1, Assembler::AVX_256bit); + vptest(rymm2, rymm2); + jcc(Assembler::notZero, VECTOR32_NOT_EQUAL);//mismatch found + addq(result, 32); + subq(length, 32); + jccb(Assembler::greaterEqual, VECTOR32_LOOP); + addq(length, 32); + jcc(Assembler::equal, SAME_TILL_END); + //falling through if less than 32 bytes left //close the branch here. + + bind(VECTOR16_TAIL); + cmpq(length, 16); + jccb(Assembler::less, VECTOR8_TAIL); + bind(VECTOR16_LOOP); + movdqu(rymm0, Address(obja, result)); + movdqu(rymm1, Address(objb, result)); + vpxor(rymm2, rymm0, rymm1, Assembler::AVX_128bit); + ptest(rymm2, rymm2); + jcc(Assembler::notZero, VECTOR16_NOT_EQUAL);//mismatch found + addq(result, 16); + subq(length, 16); + jcc(Assembler::equal, SAME_TILL_END); + //falling through if less than 16 bytes left + } else {//regular intrinsics + + cmpq(length, 16); + jccb(Assembler::less, VECTOR8_TAIL); + + subq(length, 16); + bind(VECTOR16_LOOP); + movdqu(rymm0, Address(obja, result)); + movdqu(rymm1, Address(objb, result)); + pxor(rymm0, rymm1); + ptest(rymm0, rymm0); + jcc(Assembler::notZero, VECTOR16_NOT_EQUAL);//mismatch found + addq(result, 16); + subq(length, 16); + jccb(Assembler::greaterEqual, VECTOR16_LOOP); + addq(length, 16); + jcc(Assembler::equal, SAME_TILL_END); + //falling through if less than 16 bytes left + } + + bind(VECTOR8_TAIL); + cmpq(length, 8); + jccb(Assembler::less, VECTOR4_TAIL); + bind(VECTOR8_LOOP); + movq(tmp1, Address(obja, result)); + movq(tmp2, Address(objb, result)); + xorq(tmp1, tmp2); + testq(tmp1, tmp1); + jcc(Assembler::notZero, VECTOR8_NOT_EQUAL);//mismatch found + addq(result, 8); + subq(length, 8); + jcc(Assembler::equal, SAME_TILL_END); + //falling through if less than 8 bytes left + + bind(VECTOR4_TAIL); + cmpq(length, 4); + jccb(Assembler::less, BYTES_TAIL); + bind(VECTOR4_LOOP); + movl(tmp1, Address(obja, result)); + xorl(tmp1, Address(objb, result)); + testl(tmp1, tmp1); + jcc(Assembler::notZero, VECTOR4_NOT_EQUAL);//mismatch found + addq(result, 4); + subq(length, 4); + jcc(Assembler::equal, SAME_TILL_END); + //falling through if less than 4 bytes left + + bind(BYTES_TAIL); + bind(BYTES_LOOP); + load_unsigned_byte(tmp1, Address(obja, result)); + load_unsigned_byte(tmp2, Address(objb, result)); + xorl(tmp1, tmp2); + testl(tmp1, tmp1); + jccb(Assembler::notZero, BYTES_NOT_EQUAL);//mismatch found + decq(length); + jccb(Assembler::zero, SAME_TILL_END); + incq(result); + load_unsigned_byte(tmp1, Address(obja, result)); + load_unsigned_byte(tmp2, Address(objb, result)); + xorl(tmp1, tmp2); + testl(tmp1, tmp1); + jccb(Assembler::notZero, BYTES_NOT_EQUAL);//mismatch found + decq(length); + jccb(Assembler::zero, SAME_TILL_END); + incq(result); + load_unsigned_byte(tmp1, Address(obja, result)); + load_unsigned_byte(tmp2, Address(objb, result)); + xorl(tmp1, tmp2); + testl(tmp1, tmp1); + jccb(Assembler::notZero, BYTES_NOT_EQUAL);//mismatch found + jmpb(SAME_TILL_END); + + if (UseAVX >= 2){ + bind(VECTOR32_NOT_EQUAL); + vpcmpeqb(rymm2, rymm2, rymm2, Assembler::AVX_256bit); + vpcmpeqb(rymm0, rymm0, rymm1, Assembler::AVX_256bit); + vpxor(rymm0, rymm0, rymm2, Assembler::AVX_256bit); + vpmovmskb(tmp1, rymm0); + bsfq(tmp1, tmp1); + addq(result, tmp1); + shrq(result); + jmpb(DONE); + } + + bind(VECTOR16_NOT_EQUAL); + if (UseAVX >= 2){ + vpcmpeqb(rymm2, rymm2, rymm2, Assembler::AVX_128bit); + vpcmpeqb(rymm0, rymm0, rymm1, Assembler::AVX_128bit); + pxor(rymm0, rymm2); + } else { + pcmpeqb(rymm2, rymm2); + pxor(rymm0, rymm1); + pcmpeqb(rymm0, rymm1); + pxor(rymm0, rymm2); + } + pmovmskb(tmp1, rymm0); + bsfq(tmp1, tmp1); + addq(result, tmp1); + shrq(result); + jmpb(DONE); + + bind(VECTOR8_NOT_EQUAL); + bind(VECTOR4_NOT_EQUAL); + bsfq(tmp1, tmp1); + shrq(tmp1, 3); + addq(result, tmp1); + bind(BYTES_NOT_EQUAL); + shrq(result); + jmpb(DONE); + + bind(SAME_TILL_END); + mov64(result, -1); + + bind(DONE); +} + + //Helper functions for square_to_len() /** * Store the squares of x[], right shifted one bit (divided by 2) into z[] * Preserves x and z and modifies rest of the registers. */ - void MacroAssembler::square_rshift(Register x, Register xlen, Register z, Register tmp1, Register tmp3, Register tmp4, Register tmp5, Register rdxReg, Register raxReg) { // Perform square and right shift by 1 // Handle odd xlen case first, then for even xlen do the following diff --git a/hotspot/src/cpu/x86/vm/macroAssembler_x86.hpp b/hotspot/src/cpu/x86/vm/macroAssembler_x86.hpp index 1617abd3171..bb5782820f0 100644 --- a/hotspot/src/cpu/x86/vm/macroAssembler_x86.hpp +++ b/hotspot/src/cpu/x86/vm/macroAssembler_x86.hpp @@ -846,7 +846,7 @@ class MacroAssembler: public Assembler { void call(AddressLiteral entry); // Emit the CompiledIC call idiom - void ic_call(address entry); + void ic_call(address entry, jint method_index = 0); // Jumps @@ -911,24 +911,19 @@ class MacroAssembler: public Assembler { void fast_log(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xmm2, XMMRegister xmm3, XMMRegister xmm4, XMMRegister xmm5, XMMRegister xmm6, XMMRegister xmm7, Register rax, Register rcx, Register rdx, Register tmp1 LP64_ONLY(COMMA Register tmp2)); + void fast_pow(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xmm2, XMMRegister xmm3, XMMRegister xmm4, + XMMRegister xmm5, XMMRegister xmm6, XMMRegister xmm7, Register rax, Register rcx, + Register rdx NOT_LP64(COMMA Register tmp) LP64_ONLY(COMMA Register tmp1) + LP64_ONLY(COMMA Register tmp2) LP64_ONLY(COMMA Register tmp3) LP64_ONLY(COMMA Register tmp4)); void increase_precision(); void restore_precision(); - // computes pow(x,y). Fallback to runtime call included. - void pow_with_fallback(int num_fpu_regs_in_use) { pow_or_exp(num_fpu_regs_in_use); } - private: // call runtime as a fallback for trig functions and pow/exp. void fp_runtime_fallback(address runtime_entry, int nb_args, int num_fpu_regs_in_use); - // computes 2^(Ylog2X); Ylog2X in ST(0) - void pow_exp_core_encoding(); - - // computes pow(x,y) or exp(x). Fallback to runtime call included. - void pow_or_exp(int num_fpu_regs_in_use); - // these are private because users should be doing movflt/movdbl void movss(Address dst, XMMRegister src) { Assembler::movss(dst, src); } @@ -1342,7 +1337,6 @@ public: Register carry2); void multiply_to_len(Register x, Register xlen, Register y, Register ylen, Register z, Register zlen, Register tmp1, Register tmp2, Register tmp3, Register tmp4, Register tmp5); - void square_rshift(Register x, Register len, Register z, Register tmp1, Register tmp3, Register tmp4, Register tmp5, Register rdxReg, Register raxReg); void multiply_add_64_bmi2(Register sum, Register op1, Register op2, Register carry, @@ -1361,6 +1355,9 @@ public: void mul_add(Register out, Register in, Register offset, Register len, Register k, Register tmp1, Register tmp2, Register tmp3, Register tmp4, Register tmp5, Register rdxReg, Register raxReg); + void vectorized_mismatch(Register obja, Register objb, Register length, Register log2_array_indxscale, + Register result, Register tmp1, Register tmp2, + XMMRegister vec1, XMMRegister vec2, XMMRegister vec3); #endif // CRC32 code for java.util.zip.CRC32::updateBytes() intrinsic. diff --git a/hotspot/src/cpu/x86/vm/macroAssembler_x86_libm.cpp b/hotspot/src/cpu/x86/vm/macroAssembler_x86_libm.cpp index 860f2bcd26d..e94f1d7136f 100644 --- a/hotspot/src/cpu/x86/vm/macroAssembler_x86_libm.cpp +++ b/hotspot/src/cpu/x86/vm/macroAssembler_x86_libm.cpp @@ -35,6 +35,8 @@ #define ALIGNED_(x) __attribute__ ((aligned(x))) #endif +// The 32 bit and 64 bit code is at most SSE2 compliant + /******************************************************************************/ // ALGORITHM DESCRIPTION - EXP() // --------------------- @@ -409,7 +411,7 @@ void MacroAssembler::fast_exp(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xm addq(rsp, 24); } -#endif +#endif // _LP64 #ifndef _LP64 @@ -674,7 +676,7 @@ void MacroAssembler::fast_exp(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xm movl(tmp, Address(rsp, 64)); } -#endif +#endif // !_LP64 /******************************************************************************/ // ALGORITHM DESCRIPTION - LOG() @@ -889,13 +891,23 @@ void MacroAssembler::fast_log(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xm addsd(xmm1, xmm5); movdqu(xmm2, ExternalAddress(32 + coeff)); // 0x9999999aUL, 0x3fc99999UL, 0x00000000UL, 0xbfe00000UL mulsd(xmm6, xmm7); - movddup(xmm5, xmm1); + if (VM_Version::supports_sse3()) { + movddup(xmm5, xmm1); + } else { + movdqu(xmm5, xmm1); + movlhps(xmm5, xmm5); + } mulsd(xmm7, ExternalAddress(8 + log2)); // 0x93c76730UL, 0x3ceef357UL mulsd(xmm3, xmm1); addsd(xmm0, xmm6); mulpd(xmm4, xmm5); mulpd(xmm5, xmm5); - movddup(xmm6, xmm0); + if (VM_Version::supports_sse3()) { + movddup(xmm6, xmm0); + } else { + movdqu(xmm6, xmm0); + movlhps(xmm6, xmm6); + } addsd(xmm0, xmm1); addpd(xmm4, xmm2); mulpd(xmm3, xmm5); @@ -995,7 +1007,7 @@ void MacroAssembler::fast_log(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xm addq(rsp, 24); } -#endif +#endif // _LP64 #ifndef _LP64 @@ -1285,4 +1297,3561 @@ void MacroAssembler::fast_log(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xm movl(tmp, Address(rsp, 40)); } -#endif +#endif // !_LP64 + +/******************************************************************************/ +// ALGORITHM DESCRIPTION - POW() +// --------------------- +// +// Let x=2^k * mx, mx in [1,2) +// +// log2(x) calculation: +// +// Get B~1/mx based on the output of rcpps instruction (B0) +// B = int((B0*LH*2^9+0.5))/2^9 +// LH is a short approximation for log2(e) +// +// Reduced argument, scaled by LH: +// r=B*mx-LH (computed accurately in high and low parts) +// +// log2(x) result: k - log2(B) + p(r) +// p(r) is a degree 8 polynomial +// -log2(B) read from data table (high, low parts) +// log2(x) is formed from high and low parts +// For |x| in [1-1/32, 1+1/16), a slower but more accurate computation +// based om the same table design is performed. +// +// Main path is taken if | floor(log2(|log2(|x|)|) + floor(log2|y|) | < 8, +// to filter out all potential OF/UF cases. +// exp2(y*log2(x)) is computed using an 8-bit index table and a degree 5 +// polynomial +// +// Special cases: +// pow(-0,y) = -INF and raises the divide-by-zero exception for y an odd +// integer < 0. +// pow(-0,y) = +INF and raises the divide-by-zero exception for y < 0 and +// not an odd integer. +// pow(-0,y) = -0 for y an odd integer > 0. +// pow(-0,y) = +0 for y > 0 and not an odd integer. +// pow(-1,-INF) = NaN. +// pow(+1,y) = NaN for any y, even a NaN. +// pow(x,-0) = 1 for any x, even a NaN. +// pow(x,y) = a NaN and raises the invalid exception for finite x < 0 and +// finite non-integer y. +// pow(x,-INF) = +INF for |x|<1. +// pow(x,-INF) = +0 for |x|>1. +// pow(x,+INF) = +0 for |x|<1. +// pow(x,+INF) = +INF for |x|>1. +// pow(-INF,y) = -0 for y an odd integer < 0. +// pow(-INF,y) = +0 for y < 0 and not an odd integer. +// pow(-INF,y) = -INF for y an odd integer > 0. +// pow(-INF,y) = +INF for y > 0 and not an odd integer. +// pow(+INF,y) = +0 for y <0. +// pow(+INF,y) = +INF for y >0. +// +/******************************************************************************/ + +#ifdef _LP64 +ALIGNED_(16) juint _HIGHSIGMASK[] = +{ + 0x00000000UL, 0xfffff800UL, 0x00000000UL, 0xfffff800UL +}; + +ALIGNED_(16) juint _LOG2_E[] = +{ + 0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL +}; + +ALIGNED_(16) juint _HIGHMASK_Y[] = +{ + 0x00000000UL, 0xfffffff8UL, 0x00000000UL, 0xffffffffUL +}; + +ALIGNED_(16) juint _T_exp[] = +{ + 0x00000000UL, 0x3ff00000UL, 0x00000000UL, 0x3b700000UL, 0xfa5abcbfUL, + 0x3ff00b1aUL, 0xa7609f71UL, 0xbc84f6b2UL, 0xa9fb3335UL, 0x3ff0163dUL, + 0x9ab8cdb7UL, 0x3c9b6129UL, 0x143b0281UL, 0x3ff02168UL, 0x0fc54eb6UL, + 0xbc82bf31UL, 0x3e778061UL, 0x3ff02c9aUL, 0x535b085dUL, 0xbc719083UL, + 0x2e11bbccUL, 0x3ff037d4UL, 0xeeade11aUL, 0x3c656811UL, 0xe86e7f85UL, + 0x3ff04315UL, 0x1977c96eUL, 0xbc90a31cUL, 0x72f654b1UL, 0x3ff04e5fUL, + 0x3aa0d08cUL, 0x3c84c379UL, 0xd3158574UL, 0x3ff059b0UL, 0xa475b465UL, + 0x3c8d73e2UL, 0x0e3c1f89UL, 0x3ff0650aUL, 0x5799c397UL, 0xbc95cb7bUL, + 0x29ddf6deUL, 0x3ff0706bUL, 0xe2b13c27UL, 0xbc8c91dfUL, 0x2b72a836UL, + 0x3ff07bd4UL, 0x54458700UL, 0x3c832334UL, 0x18759bc8UL, 0x3ff08745UL, + 0x4bb284ffUL, 0x3c6186beUL, 0xf66607e0UL, 0x3ff092bdUL, 0x800a3fd1UL, + 0xbc968063UL, 0xcac6f383UL, 0x3ff09e3eUL, 0x18316136UL, 0x3c914878UL, + 0x9b1f3919UL, 0x3ff0a9c7UL, 0x873d1d38UL, 0x3c85d16cUL, 0x6cf9890fUL, + 0x3ff0b558UL, 0x4adc610bUL, 0x3c98a62eUL, 0x45e46c85UL, 0x3ff0c0f1UL, + 0x06d21cefUL, 0x3c94f989UL, 0x2b7247f7UL, 0x3ff0cc92UL, 0x16e24f71UL, + 0x3c901edcUL, 0x23395decUL, 0x3ff0d83bUL, 0xe43f316aUL, 0xbc9bc14dUL, + 0x32d3d1a2UL, 0x3ff0e3ecUL, 0x27c57b52UL, 0x3c403a17UL, 0x5fdfa9c5UL, + 0x3ff0efa5UL, 0xbc54021bUL, 0xbc949db9UL, 0xaffed31bUL, 0x3ff0fb66UL, + 0xc44ebd7bUL, 0xbc6b9bedUL, 0x28d7233eUL, 0x3ff10730UL, 0x1692fdd5UL, + 0x3c8d46ebUL, 0xd0125b51UL, 0x3ff11301UL, 0x39449b3aUL, 0xbc96c510UL, + 0xab5e2ab6UL, 0x3ff11edbUL, 0xf703fb72UL, 0xbc9ca454UL, 0xc06c31ccUL, + 0x3ff12abdUL, 0xb36ca5c7UL, 0xbc51b514UL, 0x14f204abUL, 0x3ff136a8UL, + 0xba48dcf0UL, 0xbc67108fUL, 0xaea92de0UL, 0x3ff1429aUL, 0x9af1369eUL, + 0xbc932fbfUL, 0x934f312eUL, 0x3ff14e95UL, 0x39bf44abUL, 0xbc8b91e8UL, + 0xc8a58e51UL, 0x3ff15a98UL, 0xb9eeab0aUL, 0x3c82406aUL, 0x5471c3c2UL, + 0x3ff166a4UL, 0x82ea1a32UL, 0x3c58f23bUL, 0x3c7d517bUL, 0x3ff172b8UL, + 0xb9d78a76UL, 0xbc819041UL, 0x8695bbc0UL, 0x3ff17ed4UL, 0xe2ac5a64UL, + 0x3c709e3fUL, 0x388c8deaUL, 0x3ff18af9UL, 0xd1970f6cUL, 0xbc911023UL, + 0x58375d2fUL, 0x3ff19726UL, 0x85f17e08UL, 0x3c94aaddUL, 0xeb6fcb75UL, + 0x3ff1a35bUL, 0x7b4968e4UL, 0x3c8e5b4cUL, 0xf8138a1cUL, 0x3ff1af99UL, + 0xa4b69280UL, 0x3c97bf85UL, 0x84045cd4UL, 0x3ff1bbe0UL, 0x352ef607UL, + 0xbc995386UL, 0x95281c6bUL, 0x3ff1c82fUL, 0x8010f8c9UL, 0x3c900977UL, + 0x3168b9aaUL, 0x3ff1d487UL, 0x00a2643cUL, 0x3c9e016eUL, 0x5eb44027UL, + 0x3ff1e0e7UL, 0x088cb6deUL, 0xbc96fdd8UL, 0x22fcd91dUL, 0x3ff1ed50UL, + 0x027bb78cUL, 0xbc91df98UL, 0x8438ce4dUL, 0x3ff1f9c1UL, 0xa097af5cUL, + 0xbc9bf524UL, 0x88628cd6UL, 0x3ff2063bUL, 0x814a8495UL, 0x3c8dc775UL, + 0x3578a819UL, 0x3ff212beUL, 0x2cfcaac9UL, 0x3c93592dUL, 0x917ddc96UL, + 0x3ff21f49UL, 0x9494a5eeUL, 0x3c82a97eUL, 0xa27912d1UL, 0x3ff22bddUL, + 0x5577d69fUL, 0x3c8d34fbUL, 0x6e756238UL, 0x3ff2387aUL, 0xb6c70573UL, + 0x3c99b07eUL, 0xfb82140aUL, 0x3ff2451fUL, 0x911ca996UL, 0x3c8acfccUL, + 0x4fb2a63fUL, 0x3ff251ceUL, 0xbef4f4a4UL, 0x3c8ac155UL, 0x711ece75UL, + 0x3ff25e85UL, 0x4ac31b2cUL, 0x3c93e1a2UL, 0x65e27cddUL, 0x3ff26b45UL, + 0x9940e9d9UL, 0x3c82bd33UL, 0x341ddf29UL, 0x3ff2780eUL, 0x05f9e76cUL, + 0x3c9e067cUL, 0xe1f56381UL, 0x3ff284dfUL, 0x8c3f0d7eUL, 0xbc9a4c3aUL, + 0x7591bb70UL, 0x3ff291baUL, 0x28401cbdUL, 0xbc82cc72UL, 0xf51fdee1UL, + 0x3ff29e9dUL, 0xafad1255UL, 0x3c8612e8UL, 0x66d10f13UL, 0x3ff2ab8aUL, + 0x191690a7UL, 0xbc995743UL, 0xd0dad990UL, 0x3ff2b87fUL, 0xd6381aa4UL, + 0xbc410adcUL, 0x39771b2fUL, 0x3ff2c57eUL, 0xa6eb5124UL, 0xbc950145UL, + 0xa6e4030bUL, 0x3ff2d285UL, 0x54db41d5UL, 0x3c900247UL, 0x1f641589UL, + 0x3ff2df96UL, 0xfbbce198UL, 0x3c9d16cfUL, 0xa93e2f56UL, 0x3ff2ecafUL, + 0x45d52383UL, 0x3c71ca0fUL, 0x4abd886bUL, 0x3ff2f9d2UL, 0x532bda93UL, + 0xbc653c55UL, 0x0a31b715UL, 0x3ff306feUL, 0xd23182e4UL, 0x3c86f46aUL, + 0xedeeb2fdUL, 0x3ff31432UL, 0xf3f3fcd1UL, 0x3c8959a3UL, 0xfc4cd831UL, + 0x3ff32170UL, 0x8e18047cUL, 0x3c8a9ce7UL, 0x3ba8ea32UL, 0x3ff32eb8UL, + 0x3cb4f318UL, 0xbc9c45e8UL, 0xb26416ffUL, 0x3ff33c08UL, 0x843659a6UL, + 0x3c932721UL, 0x66e3fa2dUL, 0x3ff34962UL, 0x930881a4UL, 0xbc835a75UL, + 0x5f929ff1UL, 0x3ff356c5UL, 0x5c4e4628UL, 0xbc8b5ceeUL, 0xa2de883bUL, + 0x3ff36431UL, 0xa06cb85eUL, 0xbc8c3144UL, 0x373aa9cbUL, 0x3ff371a7UL, + 0xbf42eae2UL, 0xbc963aeaUL, 0x231e754aUL, 0x3ff37f26UL, 0x9eceb23cUL, + 0xbc99f5caUL, 0x6d05d866UL, 0x3ff38caeUL, 0x3c9904bdUL, 0xbc9e958dUL, + 0x1b7140efUL, 0x3ff39a40UL, 0xfc8e2934UL, 0xbc99a9a5UL, 0x34e59ff7UL, + 0x3ff3a7dbUL, 0xd661f5e3UL, 0xbc75e436UL, 0xbfec6cf4UL, 0x3ff3b57fUL, + 0xe26fff18UL, 0x3c954c66UL, 0xc313a8e5UL, 0x3ff3c32dUL, 0x375d29c3UL, + 0xbc9efff8UL, 0x44ede173UL, 0x3ff3d0e5UL, 0x8c284c71UL, 0x3c7fe8d0UL, + 0x4c123422UL, 0x3ff3dea6UL, 0x11f09ebcUL, 0x3c8ada09UL, 0xdf1c5175UL, + 0x3ff3ec70UL, 0x7b8c9bcaUL, 0xbc8af663UL, 0x04ac801cUL, 0x3ff3fa45UL, + 0xf956f9f3UL, 0xbc97d023UL, 0xc367a024UL, 0x3ff40822UL, 0xb6f4d048UL, + 0x3c8bddf8UL, 0x21f72e2aUL, 0x3ff4160aUL, 0x1c309278UL, 0xbc5ef369UL, + 0x2709468aUL, 0x3ff423fbUL, 0xc0b314ddUL, 0xbc98462dUL, 0xd950a897UL, + 0x3ff431f5UL, 0xe35f7999UL, 0xbc81c7ddUL, 0x3f84b9d4UL, 0x3ff43ffaUL, + 0x9704c003UL, 0x3c8880beUL, 0x6061892dUL, 0x3ff44e08UL, 0x04ef80d0UL, + 0x3c489b7aUL, 0x42a7d232UL, 0x3ff45c20UL, 0x82fb1f8eUL, 0xbc686419UL, + 0xed1d0057UL, 0x3ff46a41UL, 0xd1648a76UL, 0x3c9c944bUL, 0x668b3237UL, + 0x3ff4786dUL, 0xed445733UL, 0xbc9c20f0UL, 0xb5c13cd0UL, 0x3ff486a2UL, + 0xb69062f0UL, 0x3c73c1a3UL, 0xe192aed2UL, 0x3ff494e1UL, 0x5e499ea0UL, + 0xbc83b289UL, 0xf0d7d3deUL, 0x3ff4a32aUL, 0xf3d1be56UL, 0x3c99cb62UL, + 0xea6db7d7UL, 0x3ff4b17dUL, 0x7f2897f0UL, 0xbc8125b8UL, 0xd5362a27UL, + 0x3ff4bfdaUL, 0xafec42e2UL, 0x3c7d4397UL, 0xb817c114UL, 0x3ff4ce41UL, + 0x690abd5dUL, 0x3c905e29UL, 0x99fddd0dUL, 0x3ff4dcb2UL, 0xbc6a7833UL, + 0x3c98ecdbUL, 0x81d8abffUL, 0x3ff4eb2dUL, 0x2e5d7a52UL, 0xbc95257dUL, + 0x769d2ca7UL, 0x3ff4f9b2UL, 0xd25957e3UL, 0xbc94b309UL, 0x7f4531eeUL, + 0x3ff50841UL, 0x49b7465fUL, 0x3c7a249bUL, 0xa2cf6642UL, 0x3ff516daUL, + 0x69bd93efUL, 0xbc8f7685UL, 0xe83f4eefUL, 0x3ff5257dUL, 0x43efef71UL, + 0xbc7c998dUL, 0x569d4f82UL, 0x3ff5342bUL, 0x1db13cadUL, 0xbc807abeUL, + 0xf4f6ad27UL, 0x3ff542e2UL, 0x192d5f7eUL, 0x3c87926dUL, 0xca5d920fUL, + 0x3ff551a4UL, 0xefede59bUL, 0xbc8d689cUL, 0xdde910d2UL, 0x3ff56070UL, + 0x168eebf0UL, 0xbc90fb6eUL, 0x36b527daUL, 0x3ff56f47UL, 0x011d93adUL, + 0x3c99bb2cUL, 0xdbe2c4cfUL, 0x3ff57e27UL, 0x8a57b9c4UL, 0xbc90b98cUL, + 0xd497c7fdUL, 0x3ff58d12UL, 0x5b9a1de8UL, 0x3c8295e1UL, 0x27ff07ccUL, + 0x3ff59c08UL, 0xe467e60fUL, 0xbc97e2ceUL, 0xdd485429UL, 0x3ff5ab07UL, + 0x054647adUL, 0x3c96324cUL, 0xfba87a03UL, 0x3ff5ba11UL, 0x4c233e1aUL, + 0xbc9b77a1UL, 0x8a5946b7UL, 0x3ff5c926UL, 0x816986a2UL, 0x3c3c4b1bUL, + 0x90998b93UL, 0x3ff5d845UL, 0xa8b45643UL, 0xbc9cd6a7UL, 0x15ad2148UL, + 0x3ff5e76fUL, 0x3080e65eUL, 0x3c9ba6f9UL, 0x20dceb71UL, 0x3ff5f6a3UL, + 0xe3cdcf92UL, 0xbc89eaddUL, 0xb976dc09UL, 0x3ff605e1UL, 0x9b56de47UL, + 0xbc93e242UL, 0xe6cdf6f4UL, 0x3ff6152aUL, 0x4ab84c27UL, 0x3c9e4b3eUL, + 0xb03a5585UL, 0x3ff6247eUL, 0x7e40b497UL, 0xbc9383c1UL, 0x1d1929fdUL, + 0x3ff633ddUL, 0xbeb964e5UL, 0x3c984710UL, 0x34ccc320UL, 0x3ff64346UL, + 0x759d8933UL, 0xbc8c483cUL, 0xfebc8fb7UL, 0x3ff652b9UL, 0xc9a73e09UL, + 0xbc9ae3d5UL, 0x82552225UL, 0x3ff66238UL, 0x87591c34UL, 0xbc9bb609UL, + 0xc70833f6UL, 0x3ff671c1UL, 0x586c6134UL, 0xbc8e8732UL, 0xd44ca973UL, + 0x3ff68155UL, 0x44f73e65UL, 0x3c6038aeUL, 0xb19e9538UL, 0x3ff690f4UL, + 0x9aeb445dUL, 0x3c8804bdUL, 0x667f3bcdUL, 0x3ff6a09eUL, 0x13b26456UL, + 0xbc9bdd34UL, 0xfa75173eUL, 0x3ff6b052UL, 0x2c9a9d0eUL, 0x3c7a38f5UL, + 0x750bdabfUL, 0x3ff6c012UL, 0x67ff0b0dUL, 0xbc728956UL, 0xddd47645UL, + 0x3ff6cfdcUL, 0xb6f17309UL, 0x3c9c7aa9UL, 0x3c651a2fUL, 0x3ff6dfb2UL, + 0x683c88abUL, 0xbc6bbe3aUL, 0x98593ae5UL, 0x3ff6ef92UL, 0x9e1ac8b2UL, + 0xbc90b974UL, 0xf9519484UL, 0x3ff6ff7dUL, 0x25860ef6UL, 0xbc883c0fUL, + 0x66f42e87UL, 0x3ff70f74UL, 0xd45aa65fUL, 0x3c59d644UL, 0xe8ec5f74UL, + 0x3ff71f75UL, 0x86887a99UL, 0xbc816e47UL, 0x86ead08aUL, 0x3ff72f82UL, + 0x2cd62c72UL, 0xbc920aa0UL, 0x48a58174UL, 0x3ff73f9aUL, 0x6c65d53cUL, + 0xbc90a8d9UL, 0x35d7cbfdUL, 0x3ff74fbdUL, 0x618a6e1cUL, 0x3c9047fdUL, + 0x564267c9UL, 0x3ff75febUL, 0x57316dd3UL, 0xbc902459UL, 0xb1ab6e09UL, + 0x3ff77024UL, 0x169147f8UL, 0x3c9b7877UL, 0x4fde5d3fUL, 0x3ff78069UL, + 0x0a02162dUL, 0x3c9866b8UL, 0x38ac1cf6UL, 0x3ff790b9UL, 0x62aadd3eUL, + 0x3c9349a8UL, 0x73eb0187UL, 0x3ff7a114UL, 0xee04992fUL, 0xbc841577UL, + 0x0976cfdbUL, 0x3ff7b17bUL, 0x8468dc88UL, 0xbc9bebb5UL, 0x0130c132UL, + 0x3ff7c1edUL, 0xd1164dd6UL, 0x3c9f124cUL, 0x62ff86f0UL, 0x3ff7d26aUL, + 0xfb72b8b4UL, 0x3c91bddbUL, 0x36cf4e62UL, 0x3ff7e2f3UL, 0xba15797eUL, + 0x3c705d02UL, 0x8491c491UL, 0x3ff7f387UL, 0xcf9311aeUL, 0xbc807f11UL, + 0x543e1a12UL, 0x3ff80427UL, 0x626d972bUL, 0xbc927c86UL, 0xadd106d9UL, + 0x3ff814d2UL, 0x0d151d4dUL, 0x3c946437UL, 0x994cce13UL, 0x3ff82589UL, + 0xd41532d8UL, 0xbc9d4c1dUL, 0x1eb941f7UL, 0x3ff8364cUL, 0x31df2bd5UL, + 0x3c999b9aUL, 0x4623c7adUL, 0x3ff8471aUL, 0xa341cdfbUL, 0xbc88d684UL, + 0x179f5b21UL, 0x3ff857f4UL, 0xf8b216d0UL, 0xbc5ba748UL, 0x9b4492edUL, + 0x3ff868d9UL, 0x9bd4f6baUL, 0xbc9fc6f8UL, 0xd931a436UL, 0x3ff879caUL, + 0xd2db47bdUL, 0x3c85d2d7UL, 0xd98a6699UL, 0x3ff88ac7UL, 0xf37cb53aUL, + 0x3c9994c2UL, 0xa478580fUL, 0x3ff89bd0UL, 0x4475202aUL, 0x3c9d5395UL, + 0x422aa0dbUL, 0x3ff8ace5UL, 0x56864b27UL, 0x3c96e9f1UL, 0xbad61778UL, + 0x3ff8be05UL, 0xfc43446eUL, 0x3c9ecb5eUL, 0x16b5448cUL, 0x3ff8cf32UL, + 0x32e9e3aaUL, 0xbc70d55eUL, 0x5e0866d9UL, 0x3ff8e06aUL, 0x6fc9b2e6UL, + 0xbc97114aUL, 0x99157736UL, 0x3ff8f1aeUL, 0xa2e3976cUL, 0x3c85cc13UL, + 0xd0282c8aUL, 0x3ff902feUL, 0x85fe3fd2UL, 0x3c9592caUL, 0x0b91ffc6UL, + 0x3ff9145bUL, 0x2e582524UL, 0xbc9dd679UL, 0x53aa2fe2UL, 0x3ff925c3UL, + 0xa639db7fUL, 0xbc83455fUL, 0xb0cdc5e5UL, 0x3ff93737UL, 0x81b57ebcUL, + 0xbc675fc7UL, 0x2b5f98e5UL, 0x3ff948b8UL, 0x797d2d99UL, 0xbc8dc3d6UL, + 0xcbc8520fUL, 0x3ff95a44UL, 0x96a5f039UL, 0xbc764b7cUL, 0x9a7670b3UL, + 0x3ff96bddUL, 0x7f19c896UL, 0xbc5ba596UL, 0x9fde4e50UL, 0x3ff97d82UL, + 0x7c1b85d1UL, 0xbc9d185bUL, 0xe47a22a2UL, 0x3ff98f33UL, 0xa24c78ecUL, + 0x3c7cabdaUL, 0x70ca07baUL, 0x3ff9a0f1UL, 0x91cee632UL, 0xbc9173bdUL, + 0x4d53fe0dUL, 0x3ff9b2bbUL, 0x4df6d518UL, 0xbc9dd84eUL, 0x82a3f090UL, + 0x3ff9c491UL, 0xb071f2beUL, 0x3c7c7c46UL, 0x194bb8d5UL, 0x3ff9d674UL, + 0xa3dd8233UL, 0xbc9516beUL, 0x19e32323UL, 0x3ff9e863UL, 0x78e64c6eUL, + 0x3c7824caUL, 0x8d07f29eUL, 0x3ff9fa5eUL, 0xaaf1faceUL, 0xbc84a9ceUL, + 0x7b5de565UL, 0x3ffa0c66UL, 0x5d1cd533UL, 0xbc935949UL, 0xed8eb8bbUL, + 0x3ffa1e7aUL, 0xee8be70eUL, 0x3c9c6618UL, 0xec4a2d33UL, 0x3ffa309bUL, + 0x7ddc36abUL, 0x3c96305cUL, 0x80460ad8UL, 0x3ffa42c9UL, 0x589fb120UL, + 0xbc9aa780UL, 0xb23e255dUL, 0x3ffa5503UL, 0xdb8d41e1UL, 0xbc9d2f6eUL, + 0x8af46052UL, 0x3ffa674aUL, 0x30670366UL, 0x3c650f56UL, 0x1330b358UL, + 0x3ffa799eUL, 0xcac563c7UL, 0x3c9bcb7eUL, 0x53c12e59UL, 0x3ffa8bfeUL, + 0xb2ba15a9UL, 0xbc94f867UL, 0x5579fdbfUL, 0x3ffa9e6bUL, 0x0ef7fd31UL, + 0x3c90fac9UL, 0x21356ebaUL, 0x3ffab0e5UL, 0xdae94545UL, 0x3c889c31UL, + 0xbfd3f37aUL, 0x3ffac36bUL, 0xcae76cd0UL, 0xbc8f9234UL, 0x3a3c2774UL, + 0x3ffad5ffUL, 0xb6b1b8e5UL, 0x3c97ef3bUL, 0x995ad3adUL, 0x3ffae89fUL, + 0x345dcc81UL, 0x3c97a1cdUL, 0xe622f2ffUL, 0x3ffafb4cUL, 0x0f315ecdUL, + 0xbc94b2fcUL, 0x298db666UL, 0x3ffb0e07UL, 0x4c80e425UL, 0xbc9bdef5UL, + 0x6c9a8952UL, 0x3ffb20ceUL, 0x4a0756ccUL, 0x3c94dd02UL, 0xb84f15fbUL, + 0x3ffb33a2UL, 0x3084d708UL, 0xbc62805eUL, 0x15b749b1UL, 0x3ffb4684UL, + 0xe9df7c90UL, 0xbc7f763dUL, 0x8de5593aUL, 0x3ffb5972UL, 0xbbba6de3UL, + 0xbc9c71dfUL, 0x29f1c52aUL, 0x3ffb6c6eUL, 0x52883f6eUL, 0x3c92a8f3UL, + 0xf2fb5e47UL, 0x3ffb7f76UL, 0x7e54ac3bUL, 0xbc75584fUL, 0xf22749e4UL, + 0x3ffb928cUL, 0x54cb65c6UL, 0xbc9b7216UL, 0x30a1064aUL, 0x3ffba5b0UL, + 0x0e54292eUL, 0xbc9efcd3UL, 0xb79a6f1fUL, 0x3ffbb8e0UL, 0xc9696205UL, + 0xbc3f52d1UL, 0x904bc1d2UL, 0x3ffbcc1eUL, 0x7a2d9e84UL, 0x3c823dd0UL, + 0xc3f3a207UL, 0x3ffbdf69UL, 0x60ea5b53UL, 0xbc3c2623UL, 0x5bd71e09UL, + 0x3ffbf2c2UL, 0x3f6b9c73UL, 0xbc9efdcaUL, 0x6141b33dUL, 0x3ffc0628UL, + 0xa1fbca34UL, 0xbc8d8a5aUL, 0xdd85529cUL, 0x3ffc199bUL, 0x895048ddUL, + 0x3c811065UL, 0xd9fa652cUL, 0x3ffc2d1cUL, 0x17c8a5d7UL, 0xbc96e516UL, + 0x5fffd07aUL, 0x3ffc40abUL, 0xe083c60aUL, 0x3c9b4537UL, 0x78fafb22UL, + 0x3ffc5447UL, 0x2493b5afUL, 0x3c912f07UL, 0x2e57d14bUL, 0x3ffc67f1UL, + 0xff483cadUL, 0x3c92884dUL, 0x8988c933UL, 0x3ffc7ba8UL, 0xbe255559UL, + 0xbc8e76bbUL, 0x9406e7b5UL, 0x3ffc8f6dUL, 0x48805c44UL, 0x3c71acbcUL, + 0x5751c4dbUL, 0x3ffca340UL, 0xd10d08f5UL, 0xbc87f2beUL, 0xdcef9069UL, + 0x3ffcb720UL, 0xd1e949dbUL, 0x3c7503cbUL, 0x2e6d1675UL, 0x3ffccb0fUL, + 0x86009092UL, 0xbc7d220fUL, 0x555dc3faUL, 0x3ffcdf0bUL, 0x53829d72UL, + 0xbc8dd83bUL, 0x5b5bab74UL, 0x3ffcf315UL, 0xb86dff57UL, 0xbc9a08e9UL, + 0x4a07897cUL, 0x3ffd072dUL, 0x43797a9cUL, 0xbc9cbc37UL, 0x2b08c968UL, + 0x3ffd1b53UL, 0x219a36eeUL, 0x3c955636UL, 0x080d89f2UL, 0x3ffd2f87UL, + 0x719d8578UL, 0xbc9d487bUL, 0xeacaa1d6UL, 0x3ffd43c8UL, 0xbf5a1614UL, + 0x3c93db53UL, 0xdcfba487UL, 0x3ffd5818UL, 0xd75b3707UL, 0x3c82ed02UL, + 0xe862e6d3UL, 0x3ffd6c76UL, 0x4a8165a0UL, 0x3c5fe87aUL, 0x16c98398UL, + 0x3ffd80e3UL, 0x8beddfe8UL, 0xbc911ec1UL, 0x71ff6075UL, 0x3ffd955dUL, + 0xbb9af6beUL, 0x3c9a052dUL, 0x03db3285UL, 0x3ffda9e6UL, 0x696db532UL, + 0x3c9c2300UL, 0xd63a8315UL, 0x3ffdbe7cUL, 0x926b8be4UL, 0xbc9b76f1UL, + 0xf301b460UL, 0x3ffdd321UL, 0x78f018c3UL, 0x3c92da57UL, 0x641c0658UL, + 0x3ffde7d5UL, 0x8e79ba8fUL, 0xbc9ca552UL, 0x337b9b5fUL, 0x3ffdfc97UL, + 0x4f184b5cUL, 0xbc91a5cdUL, 0x6b197d17UL, 0x3ffe1167UL, 0xbd5c7f44UL, + 0xbc72b529UL, 0x14f5a129UL, 0x3ffe2646UL, 0x817a1496UL, 0xbc97b627UL, + 0x3b16ee12UL, 0x3ffe3b33UL, 0x31fdc68bUL, 0xbc99f4a4UL, 0xe78b3ff6UL, + 0x3ffe502eUL, 0x80a9cc8fUL, 0x3c839e89UL, 0x24676d76UL, 0x3ffe6539UL, + 0x7522b735UL, 0xbc863ff8UL, 0xfbc74c83UL, 0x3ffe7a51UL, 0xca0c8de2UL, + 0x3c92d522UL, 0x77cdb740UL, 0x3ffe8f79UL, 0x80b054b1UL, 0xbc910894UL, + 0xa2a490daUL, 0x3ffea4afUL, 0x179c2893UL, 0xbc9e9c23UL, 0x867cca6eUL, + 0x3ffeb9f4UL, 0x2293e4f2UL, 0x3c94832fUL, 0x2d8e67f1UL, 0x3ffecf48UL, + 0xb411ad8cUL, 0xbc9c93f3UL, 0xa2188510UL, 0x3ffee4aaUL, 0xa487568dUL, + 0x3c91c68dUL, 0xee615a27UL, 0x3ffefa1bUL, 0x86a4b6b0UL, 0x3c9dc7f4UL, + 0x1cb6412aUL, 0x3fff0f9cUL, 0x65181d45UL, 0xbc932200UL, 0x376bba97UL, + 0x3fff252bUL, 0xbf0d8e43UL, 0x3c93a1a5UL, 0x48dd7274UL, 0x3fff3ac9UL, + 0x3ed837deUL, 0xbc795a5aUL, 0x5b6e4540UL, 0x3fff5076UL, 0x2dd8a18bUL, + 0x3c99d3e1UL, 0x798844f8UL, 0x3fff6632UL, 0x3539343eUL, 0x3c9fa37bUL, + 0xad9cbe14UL, 0x3fff7bfdUL, 0xd006350aUL, 0xbc9dbb12UL, 0x02243c89UL, + 0x3fff91d8UL, 0xa779f689UL, 0xbc612ea8UL, 0x819e90d8UL, 0x3fffa7c1UL, + 0xf3a5931eUL, 0x3c874853UL, 0x3692d514UL, 0x3fffbdbaUL, 0x15098eb6UL, + 0xbc796773UL, 0x2b8f71f1UL, 0x3fffd3c2UL, 0x966579e7UL, 0x3c62eb74UL, + 0x6b2a23d9UL, 0x3fffe9d9UL, 0x7442fde3UL, 0x3c74a603UL +}; + +ALIGNED_(16) juint _e_coeff[] = +{ + 0xe78a6731UL, 0x3f55d87fUL, 0xd704a0c0UL, 0x3fac6b08UL, 0x6fba4e77UL, + 0x3f83b2abUL, 0xff82c58fUL, 0x3fcebfbdUL, 0xfefa39efUL, 0x3fe62e42UL, + 0x00000000UL, 0x00000000UL +}; + +ALIGNED_(16) juint _coeff_h[] = +{ + 0x00000000UL, 0xbfd61a00UL, 0x00000000UL, 0xbf5dabe1UL +}; + +ALIGNED_(16) juint _HIGHMASK_LOG_X[] = +{ + 0xf8000000UL, 0xffffffffUL, 0x00000000UL, 0xfffff800UL +}; + +ALIGNED_(8) juint _HALFMASK[] = +{ + 0xf8000000UL, 0xffffffffUL, 0xf8000000UL, 0xffffffffUL +}; + +ALIGNED_(16) juint _coeff_pow[] = +{ + 0x6dc96112UL, 0xbf836578UL, 0xee241472UL, 0xbf9b0301UL, 0x9f95985aUL, + 0xbfb528dbUL, 0xb3841d2aUL, 0xbfd619b6UL, 0x518775e3UL, 0x3f9004f2UL, + 0xac8349bbUL, 0x3fa76c9bUL, 0x486ececcUL, 0x3fc4635eUL, 0x161bb241UL, + 0xbf5dabe1UL, 0x9f95985aUL, 0xbfb528dbUL, 0xf8b5787dUL, 0x3ef2531eUL, + 0x486ececbUL, 0x3fc4635eUL, 0x412055ccUL, 0xbdd61bb2UL +}; + +ALIGNED_(16) juint _L_tbl_pow[] = +{ + 0x00000000UL, 0x3ff00000UL, 0x00000000UL, 0x00000000UL, 0x20000000UL, + 0x3feff00aUL, 0x96621f95UL, 0x3e5b1856UL, 0xe0000000UL, 0x3fefe019UL, + 0xe5916f9eUL, 0xbe325278UL, 0x00000000UL, 0x3fefd02fUL, 0x859a1062UL, + 0x3e595fb7UL, 0xc0000000UL, 0x3fefc049UL, 0xb245f18fUL, 0xbe529c38UL, + 0xe0000000UL, 0x3fefb069UL, 0xad2880a7UL, 0xbe501230UL, 0x60000000UL, + 0x3fefa08fUL, 0xc8e72420UL, 0x3e597bd1UL, 0x80000000UL, 0x3fef90baUL, + 0xc30c4500UL, 0xbe5d6c75UL, 0xe0000000UL, 0x3fef80eaUL, 0x02c63f43UL, + 0x3e2e1318UL, 0xc0000000UL, 0x3fef7120UL, 0xb3d4ccccUL, 0xbe44c52aUL, + 0x00000000UL, 0x3fef615cUL, 0xdbd91397UL, 0xbe4e7d6cUL, 0xa0000000UL, + 0x3fef519cUL, 0x65c5cd68UL, 0xbe522dc8UL, 0xa0000000UL, 0x3fef41e2UL, + 0x46d1306cUL, 0xbe5a840eUL, 0xe0000000UL, 0x3fef322dUL, 0xd2980e94UL, + 0x3e5071afUL, 0xa0000000UL, 0x3fef227eUL, 0x773abadeUL, 0xbe5891e5UL, + 0xa0000000UL, 0x3fef12d4UL, 0xdc6bf46bUL, 0xbe5cccbeUL, 0xe0000000UL, + 0x3fef032fUL, 0xbc7247faUL, 0xbe2bab83UL, 0x80000000UL, 0x3feef390UL, + 0xbcaa1e46UL, 0xbe53bb3bUL, 0x60000000UL, 0x3feee3f6UL, 0x5f6c682dUL, + 0xbe54c619UL, 0x80000000UL, 0x3feed461UL, 0x5141e368UL, 0xbe4b6d86UL, + 0xe0000000UL, 0x3feec4d1UL, 0xec678f76UL, 0xbe369af6UL, 0x80000000UL, + 0x3feeb547UL, 0x41301f55UL, 0xbe2d4312UL, 0x60000000UL, 0x3feea5c2UL, + 0x676da6bdUL, 0xbe4d8dd0UL, 0x60000000UL, 0x3fee9642UL, 0x57a891c4UL, + 0x3e51f991UL, 0xa0000000UL, 0x3fee86c7UL, 0xe4eb491eUL, 0x3e579bf9UL, + 0x20000000UL, 0x3fee7752UL, 0xfddc4a2cUL, 0xbe3356e6UL, 0xc0000000UL, + 0x3fee67e1UL, 0xd75b5bf1UL, 0xbe449531UL, 0x80000000UL, 0x3fee5876UL, + 0xbd423b8eUL, 0x3df54fe4UL, 0x60000000UL, 0x3fee4910UL, 0x330e51b9UL, + 0x3e54289cUL, 0x80000000UL, 0x3fee39afUL, 0x8651a95fUL, 0xbe55aad6UL, + 0xa0000000UL, 0x3fee2a53UL, 0x5e98c708UL, 0xbe2fc4a9UL, 0xe0000000UL, + 0x3fee1afcUL, 0x0989328dUL, 0x3e23958cUL, 0x40000000UL, 0x3fee0babUL, + 0xee642abdUL, 0xbe425dd8UL, 0xa0000000UL, 0x3fedfc5eUL, 0xc394d236UL, + 0x3e526362UL, 0x20000000UL, 0x3feded17UL, 0xe104aa8eUL, 0x3e4ce247UL, + 0xc0000000UL, 0x3fedddd4UL, 0x265a9be4UL, 0xbe5bb77aUL, 0x40000000UL, + 0x3fedce97UL, 0x0ecac52fUL, 0x3e4a7cb1UL, 0xe0000000UL, 0x3fedbf5eUL, + 0x124cb3b8UL, 0x3e257024UL, 0x80000000UL, 0x3fedb02bUL, 0xe6d4febeUL, + 0xbe2033eeUL, 0x20000000UL, 0x3feda0fdUL, 0x39cca00eUL, 0xbe3ddabcUL, + 0xc0000000UL, 0x3fed91d3UL, 0xef8a552aUL, 0xbe543390UL, 0x40000000UL, + 0x3fed82afUL, 0xb8e85204UL, 0x3e513850UL, 0xe0000000UL, 0x3fed738fUL, + 0x3d59fe08UL, 0xbe5db728UL, 0x40000000UL, 0x3fed6475UL, 0x3aa7ead1UL, + 0x3e58804bUL, 0xc0000000UL, 0x3fed555fUL, 0xf8a35ba9UL, 0xbe5298b0UL, + 0x00000000UL, 0x3fed464fUL, 0x9a88dd15UL, 0x3e5a8cdbUL, 0x40000000UL, + 0x3fed3743UL, 0xb0b0a190UL, 0x3e598635UL, 0x80000000UL, 0x3fed283cUL, + 0xe2113295UL, 0xbe5c1119UL, 0x80000000UL, 0x3fed193aUL, 0xafbf1728UL, + 0xbe492e9cUL, 0x60000000UL, 0x3fed0a3dUL, 0xe4a4ccf3UL, 0x3e19b90eUL, + 0x20000000UL, 0x3fecfb45UL, 0xba3cbeb8UL, 0x3e406b50UL, 0xc0000000UL, + 0x3fecec51UL, 0x110f7dddUL, 0x3e0d6806UL, 0x40000000UL, 0x3fecdd63UL, + 0x7dd7d508UL, 0xbe5a8943UL, 0x80000000UL, 0x3fecce79UL, 0x9b60f271UL, + 0xbe50676aUL, 0x80000000UL, 0x3fecbf94UL, 0x0b9ad660UL, 0x3e59174fUL, + 0x60000000UL, 0x3fecb0b4UL, 0x00823d9cUL, 0x3e5bbf72UL, 0x20000000UL, + 0x3feca1d9UL, 0x38a6ec89UL, 0xbe4d38f9UL, 0x80000000UL, 0x3fec9302UL, + 0x3a0b7d8eUL, 0x3e53dbfdUL, 0xc0000000UL, 0x3fec8430UL, 0xc6826b34UL, + 0xbe27c5c9UL, 0xc0000000UL, 0x3fec7563UL, 0x0c706381UL, 0xbe593653UL, + 0x60000000UL, 0x3fec669bUL, 0x7df34ec7UL, 0x3e461ab5UL, 0xe0000000UL, + 0x3fec57d7UL, 0x40e5e7e8UL, 0xbe5c3daeUL, 0x00000000UL, 0x3fec4919UL, + 0x5602770fUL, 0xbe55219dUL, 0xc0000000UL, 0x3fec3a5eUL, 0xec7911ebUL, + 0x3e5a5d25UL, 0x60000000UL, 0x3fec2ba9UL, 0xb39ea225UL, 0xbe53c00bUL, + 0x80000000UL, 0x3fec1cf8UL, 0x967a212eUL, 0x3e5a8ddfUL, 0x60000000UL, + 0x3fec0e4cUL, 0x580798bdUL, 0x3e5f53abUL, 0x00000000UL, 0x3febffa5UL, + 0xb8282df6UL, 0xbe46b874UL, 0x20000000UL, 0x3febf102UL, 0xe33a6729UL, + 0x3e54963fUL, 0x00000000UL, 0x3febe264UL, 0x3b53e88aUL, 0xbe3adce1UL, + 0x60000000UL, 0x3febd3caUL, 0xc2585084UL, 0x3e5cde9fUL, 0x80000000UL, + 0x3febc535UL, 0xa335c5eeUL, 0xbe39fd9cUL, 0x20000000UL, 0x3febb6a5UL, + 0x7325b04dUL, 0x3e42ba15UL, 0x60000000UL, 0x3feba819UL, 0x1564540fUL, + 0x3e3a9f35UL, 0x40000000UL, 0x3feb9992UL, 0x83fff592UL, 0xbe5465ceUL, + 0xa0000000UL, 0x3feb8b0fUL, 0xb9da63d3UL, 0xbe4b1a0aUL, 0x80000000UL, + 0x3feb7c91UL, 0x6d6f1ea4UL, 0x3e557657UL, 0x00000000UL, 0x3feb6e18UL, + 0x5e80a1bfUL, 0x3e4ddbb6UL, 0x00000000UL, 0x3feb5fa3UL, 0x1c9eacb5UL, + 0x3e592877UL, 0xa0000000UL, 0x3feb5132UL, 0x6d40beb3UL, 0xbe51858cUL, + 0xa0000000UL, 0x3feb42c6UL, 0xd740c67bUL, 0x3e427ad2UL, 0x40000000UL, + 0x3feb345fUL, 0xa3e0cceeUL, 0xbe5c2fc4UL, 0x40000000UL, 0x3feb25fcUL, + 0x8e752b50UL, 0xbe3da3c2UL, 0xc0000000UL, 0x3feb179dUL, 0xa892e7deUL, + 0x3e1fb481UL, 0xc0000000UL, 0x3feb0943UL, 0x21ed71e9UL, 0xbe365206UL, + 0x20000000UL, 0x3feafaeeUL, 0x0e1380a3UL, 0x3e5c5b7bUL, 0x20000000UL, + 0x3feaec9dUL, 0x3c3d640eUL, 0xbe5dbbd0UL, 0x60000000UL, 0x3feade50UL, + 0x8f97a715UL, 0x3e3a8ec5UL, 0x20000000UL, 0x3fead008UL, 0x23ab2839UL, + 0x3e2fe98aUL, 0x40000000UL, 0x3feac1c4UL, 0xf4bbd50fUL, 0x3e54d8f6UL, + 0xe0000000UL, 0x3feab384UL, 0x14757c4dUL, 0xbe48774cUL, 0xc0000000UL, + 0x3feaa549UL, 0x7c7b0eeaUL, 0x3e5b51bbUL, 0x20000000UL, 0x3fea9713UL, + 0xf56f7013UL, 0x3e386200UL, 0xe0000000UL, 0x3fea88e0UL, 0xbe428ebeUL, + 0xbe514af5UL, 0xe0000000UL, 0x3fea7ab2UL, 0x8d0e4496UL, 0x3e4f9165UL, + 0x60000000UL, 0x3fea6c89UL, 0xdbacc5d5UL, 0xbe5c063bUL, 0x20000000UL, + 0x3fea5e64UL, 0x3f19d970UL, 0xbe5a0c8cUL, 0x20000000UL, 0x3fea5043UL, + 0x09ea3e6bUL, 0x3e5065dcUL, 0x80000000UL, 0x3fea4226UL, 0x78df246cUL, + 0x3e5e05f6UL, 0x40000000UL, 0x3fea340eUL, 0x4057d4a0UL, 0x3e431b2bUL, + 0x40000000UL, 0x3fea25faUL, 0x82867bb5UL, 0x3e4b76beUL, 0xa0000000UL, + 0x3fea17eaUL, 0x9436f40aUL, 0xbe5aad39UL, 0x20000000UL, 0x3fea09dfUL, + 0x4b5253b3UL, 0x3e46380bUL, 0x00000000UL, 0x3fe9fbd8UL, 0x8fc52466UL, + 0xbe386f9bUL, 0x20000000UL, 0x3fe9edd5UL, 0x22d3f344UL, 0xbe538347UL, + 0x60000000UL, 0x3fe9dfd6UL, 0x1ac33522UL, 0x3e5dbc53UL, 0x00000000UL, + 0x3fe9d1dcUL, 0xeabdff1dUL, 0x3e40fc0cUL, 0xe0000000UL, 0x3fe9c3e5UL, + 0xafd30e73UL, 0xbe585e63UL, 0xe0000000UL, 0x3fe9b5f3UL, 0xa52f226aUL, + 0xbe43e8f9UL, 0x20000000UL, 0x3fe9a806UL, 0xecb8698dUL, 0xbe515b36UL, + 0x80000000UL, 0x3fe99a1cUL, 0xf2b4e89dUL, 0x3e48b62bUL, 0x20000000UL, + 0x3fe98c37UL, 0x7c9a88fbUL, 0x3e44414cUL, 0x00000000UL, 0x3fe97e56UL, + 0xda015741UL, 0xbe5d13baUL, 0xe0000000UL, 0x3fe97078UL, 0x5fdace06UL, + 0x3e51b947UL, 0x00000000UL, 0x3fe962a0UL, 0x956ca094UL, 0x3e518785UL, + 0x40000000UL, 0x3fe954cbUL, 0x01164c1dUL, 0x3e5d5b57UL, 0xc0000000UL, + 0x3fe946faUL, 0xe63b3767UL, 0xbe4f84e7UL, 0x40000000UL, 0x3fe9392eUL, + 0xe57cc2a9UL, 0x3e34eda3UL, 0xe0000000UL, 0x3fe92b65UL, 0x8c75b544UL, + 0x3e5766a0UL, 0xc0000000UL, 0x3fe91da1UL, 0x37d1d087UL, 0xbe5e2ab1UL, + 0x80000000UL, 0x3fe90fe1UL, 0xa953dc20UL, 0x3e5fa1f3UL, 0x80000000UL, + 0x3fe90225UL, 0xdbd3f369UL, 0x3e47d6dbUL, 0xa0000000UL, 0x3fe8f46dUL, + 0x1c9be989UL, 0xbe5e2b0aUL, 0xa0000000UL, 0x3fe8e6b9UL, 0x3c93d76aUL, + 0x3e5c8618UL, 0xe0000000UL, 0x3fe8d909UL, 0x2182fc9aUL, 0xbe41aa9eUL, + 0x20000000UL, 0x3fe8cb5eUL, 0xe6b3539dUL, 0xbe530d19UL, 0x60000000UL, + 0x3fe8bdb6UL, 0x49e58cc3UL, 0xbe3bb374UL, 0xa0000000UL, 0x3fe8b012UL, + 0xa7cfeb8fUL, 0x3e56c412UL, 0x00000000UL, 0x3fe8a273UL, 0x8d52bc19UL, + 0x3e1429b8UL, 0x60000000UL, 0x3fe894d7UL, 0x4dc32c6cUL, 0xbe48604cUL, + 0xc0000000UL, 0x3fe8873fUL, 0x0c868e56UL, 0xbe564ee5UL, 0x00000000UL, + 0x3fe879acUL, 0x56aee828UL, 0x3e5e2fd8UL, 0x60000000UL, 0x3fe86c1cUL, + 0x7ceab8ecUL, 0x3e493365UL, 0xc0000000UL, 0x3fe85e90UL, 0x78d4dadcUL, + 0xbe4f7f25UL, 0x00000000UL, 0x3fe85109UL, 0x0ccd8280UL, 0x3e31e7a2UL, + 0x40000000UL, 0x3fe84385UL, 0x34ba4e15UL, 0x3e328077UL, 0x80000000UL, + 0x3fe83605UL, 0xa670975aUL, 0xbe53eee5UL, 0xa0000000UL, 0x3fe82889UL, + 0xf61b77b2UL, 0xbe43a20aUL, 0xa0000000UL, 0x3fe81b11UL, 0x13e6643bUL, + 0x3e5e5fe5UL, 0xc0000000UL, 0x3fe80d9dUL, 0x82cc94e8UL, 0xbe5ff1f9UL, + 0xa0000000UL, 0x3fe8002dUL, 0x8a0c9c5dUL, 0xbe42b0e7UL, 0x60000000UL, + 0x3fe7f2c1UL, 0x22a16f01UL, 0x3e5d9ea0UL, 0x20000000UL, 0x3fe7e559UL, + 0xc38cd451UL, 0x3e506963UL, 0xc0000000UL, 0x3fe7d7f4UL, 0x9902bc71UL, + 0x3e4503d7UL, 0x40000000UL, 0x3fe7ca94UL, 0xdef2a3c0UL, 0x3e3d98edUL, + 0xa0000000UL, 0x3fe7bd37UL, 0xed49abb0UL, 0x3e24c1ffUL, 0xe0000000UL, + 0x3fe7afdeUL, 0xe3b0be70UL, 0xbe40c467UL, 0x00000000UL, 0x3fe7a28aUL, + 0xaf9f193cUL, 0xbe5dff6cUL, 0xe0000000UL, 0x3fe79538UL, 0xb74cf6b6UL, + 0xbe258ed0UL, 0xa0000000UL, 0x3fe787ebUL, 0x1d9127c7UL, 0x3e345fb0UL, + 0x40000000UL, 0x3fe77aa2UL, 0x1028c21dUL, 0xbe4619bdUL, 0xa0000000UL, + 0x3fe76d5cUL, 0x7cb0b5e4UL, 0x3e40f1a2UL, 0xe0000000UL, 0x3fe7601aUL, + 0x2b1bc4adUL, 0xbe32e8bbUL, 0xe0000000UL, 0x3fe752dcUL, 0x6839f64eUL, + 0x3e41f57bUL, 0xc0000000UL, 0x3fe745a2UL, 0xc4121f7eUL, 0xbe52c40aUL, + 0x60000000UL, 0x3fe7386cUL, 0xd6852d72UL, 0xbe5c4e6bUL, 0xc0000000UL, + 0x3fe72b39UL, 0x91d690f7UL, 0xbe57f88fUL, 0xe0000000UL, 0x3fe71e0aUL, + 0x627a2159UL, 0xbe4425d5UL, 0xc0000000UL, 0x3fe710dfUL, 0x50a54033UL, + 0x3e422b7eUL, 0x60000000UL, 0x3fe703b8UL, 0x3b0b5f91UL, 0x3e5d3857UL, + 0xe0000000UL, 0x3fe6f694UL, 0x84d628a2UL, 0xbe51f090UL, 0x00000000UL, + 0x3fe6e975UL, 0x306d8894UL, 0xbe414d83UL, 0xe0000000UL, 0x3fe6dc58UL, + 0x30bf24aaUL, 0xbe4650caUL, 0x80000000UL, 0x3fe6cf40UL, 0xd4628d69UL, + 0xbe5db007UL, 0xc0000000UL, 0x3fe6c22bUL, 0xa2aae57bUL, 0xbe31d279UL, + 0xc0000000UL, 0x3fe6b51aUL, 0x860edf7eUL, 0xbe2d4c4aUL, 0x80000000UL, + 0x3fe6a80dUL, 0xf3559341UL, 0xbe5f7e98UL, 0xe0000000UL, 0x3fe69b03UL, + 0xa885899eUL, 0xbe5c2011UL, 0xe0000000UL, 0x3fe68dfdUL, 0x2bdc6d37UL, + 0x3e224a82UL, 0xa0000000UL, 0x3fe680fbUL, 0xc12ad1b9UL, 0xbe40cf56UL, + 0x00000000UL, 0x3fe673fdUL, 0x1bcdf659UL, 0xbdf52f2dUL, 0x00000000UL, + 0x3fe66702UL, 0x5df10408UL, 0x3e5663e0UL, 0xc0000000UL, 0x3fe65a0aUL, + 0xa4070568UL, 0xbe40b12fUL, 0x00000000UL, 0x3fe64d17UL, 0x71c54c47UL, + 0x3e5f5e8bUL, 0x00000000UL, 0x3fe64027UL, 0xbd4b7e83UL, 0x3e42ead6UL, + 0xa0000000UL, 0x3fe6333aUL, 0x61598bd2UL, 0xbe4c48d4UL, 0xc0000000UL, + 0x3fe62651UL, 0x6f538d61UL, 0x3e548401UL, 0xa0000000UL, 0x3fe6196cUL, + 0x14344120UL, 0xbe529af6UL, 0x00000000UL, 0x3fe60c8bUL, 0x5982c587UL, + 0xbe3e1e4fUL, 0x00000000UL, 0x3fe5ffadUL, 0xfe51d4eaUL, 0xbe4c897aUL, + 0x80000000UL, 0x3fe5f2d2UL, 0xfd46ebe1UL, 0x3e552e00UL, 0xa0000000UL, + 0x3fe5e5fbUL, 0xa4695699UL, 0x3e5ed471UL, 0x60000000UL, 0x3fe5d928UL, + 0x80d118aeUL, 0x3e456b61UL, 0xa0000000UL, 0x3fe5cc58UL, 0x304c330bUL, + 0x3e54dc29UL, 0x80000000UL, 0x3fe5bf8cUL, 0x0af2dedfUL, 0xbe3aa9bdUL, + 0xe0000000UL, 0x3fe5b2c3UL, 0x15fc9258UL, 0xbe479a37UL, 0xc0000000UL, + 0x3fe5a5feUL, 0x9292c7eaUL, 0x3e188650UL, 0x20000000UL, 0x3fe5993dUL, + 0x33b4d380UL, 0x3e5d6d93UL, 0x20000000UL, 0x3fe58c7fUL, 0x02fd16c7UL, + 0x3e2fe961UL, 0xa0000000UL, 0x3fe57fc4UL, 0x4a05edb6UL, 0xbe4d55b4UL, + 0xa0000000UL, 0x3fe5730dUL, 0x3d443abbUL, 0xbe5e6954UL, 0x00000000UL, + 0x3fe5665aUL, 0x024acfeaUL, 0x3e50e61bUL, 0x00000000UL, 0x3fe559aaUL, + 0xcc9edd09UL, 0xbe325403UL, 0x60000000UL, 0x3fe54cfdUL, 0x1fe26950UL, + 0x3e5d500eUL, 0x60000000UL, 0x3fe54054UL, 0x6c5ae164UL, 0xbe4a79b4UL, + 0xc0000000UL, 0x3fe533aeUL, 0x154b0287UL, 0xbe401571UL, 0xa0000000UL, + 0x3fe5270cUL, 0x0673f401UL, 0xbe56e56bUL, 0xe0000000UL, 0x3fe51a6dUL, + 0x751b639cUL, 0x3e235269UL, 0xa0000000UL, 0x3fe50dd2UL, 0x7c7b2bedUL, + 0x3ddec887UL, 0xc0000000UL, 0x3fe5013aUL, 0xafab4e17UL, 0x3e5e7575UL, + 0x60000000UL, 0x3fe4f4a6UL, 0x2e308668UL, 0x3e59aed6UL, 0x80000000UL, + 0x3fe4e815UL, 0xf33e2a76UL, 0xbe51f184UL, 0xe0000000UL, 0x3fe4db87UL, + 0x839f3e3eUL, 0x3e57db01UL, 0xc0000000UL, 0x3fe4cefdUL, 0xa9eda7bbUL, + 0x3e535e0fUL, 0x00000000UL, 0x3fe4c277UL, 0x2a8f66a5UL, 0x3e5ce451UL, + 0xc0000000UL, 0x3fe4b5f3UL, 0x05192456UL, 0xbe4e8518UL, 0xc0000000UL, + 0x3fe4a973UL, 0x4aa7cd1dUL, 0x3e46784aUL, 0x40000000UL, 0x3fe49cf7UL, + 0x8e23025eUL, 0xbe5749f2UL, 0x00000000UL, 0x3fe4907eUL, 0x18d30215UL, + 0x3e360f39UL, 0x20000000UL, 0x3fe48408UL, 0x63dcf2f3UL, 0x3e5e00feUL, + 0xc0000000UL, 0x3fe47795UL, 0x46182d09UL, 0xbe5173d9UL, 0xa0000000UL, + 0x3fe46b26UL, 0x8f0e62aaUL, 0xbe48f281UL, 0xe0000000UL, 0x3fe45ebaUL, + 0x5775c40cUL, 0xbe56aad4UL, 0x60000000UL, 0x3fe45252UL, 0x0fe25f69UL, + 0x3e48bd71UL, 0x40000000UL, 0x3fe445edUL, 0xe9989ec5UL, 0x3e590d97UL, + 0x80000000UL, 0x3fe4398bUL, 0xb3d9ffe3UL, 0x3e479dbcUL, 0x20000000UL, + 0x3fe42d2dUL, 0x388e4d2eUL, 0xbe5eed80UL, 0xe0000000UL, 0x3fe420d1UL, + 0x6f797c18UL, 0x3e554b4cUL, 0x20000000UL, 0x3fe4147aUL, 0x31048bb4UL, + 0xbe5b1112UL, 0x80000000UL, 0x3fe40825UL, 0x2efba4f9UL, 0x3e48ebc7UL, + 0x40000000UL, 0x3fe3fbd4UL, 0x50201119UL, 0x3e40b701UL, 0x40000000UL, + 0x3fe3ef86UL, 0x0a4db32cUL, 0x3e551de8UL, 0xa0000000UL, 0x3fe3e33bUL, + 0x0c9c148bUL, 0xbe50c1f6UL, 0x20000000UL, 0x3fe3d6f4UL, 0xc9129447UL, + 0x3e533fa0UL, 0x00000000UL, 0x3fe3cab0UL, 0xaae5b5a0UL, 0xbe22b68eUL, + 0x20000000UL, 0x3fe3be6fUL, 0x02305e8aUL, 0xbe54fc08UL, 0x60000000UL, + 0x3fe3b231UL, 0x7f908258UL, 0x3e57dc05UL, 0x00000000UL, 0x3fe3a5f7UL, + 0x1a09af78UL, 0x3e08038bUL, 0xe0000000UL, 0x3fe399bfUL, 0x490643c1UL, + 0xbe5dbe42UL, 0xe0000000UL, 0x3fe38d8bUL, 0x5e8ad724UL, 0xbe3c2b72UL, + 0x20000000UL, 0x3fe3815bUL, 0xc67196b6UL, 0x3e1713cfUL, 0xa0000000UL, + 0x3fe3752dUL, 0x6182e429UL, 0xbe3ec14cUL, 0x40000000UL, 0x3fe36903UL, + 0xab6eb1aeUL, 0x3e5a2cc5UL, 0x40000000UL, 0x3fe35cdcUL, 0xfe5dc064UL, + 0xbe5c5878UL, 0x40000000UL, 0x3fe350b8UL, 0x0ba6b9e4UL, 0x3e51619bUL, + 0x80000000UL, 0x3fe34497UL, 0x857761aaUL, 0x3e5fff53UL, 0x00000000UL, + 0x3fe3387aUL, 0xf872d68cUL, 0x3e484f4dUL, 0xa0000000UL, 0x3fe32c5fUL, + 0x087e97c2UL, 0x3e52842eUL, 0x80000000UL, 0x3fe32048UL, 0x73d6d0c0UL, + 0xbe503edfUL, 0x80000000UL, 0x3fe31434UL, 0x0c1456a1UL, 0xbe5f72adUL, + 0xa0000000UL, 0x3fe30823UL, 0x83a1a4d5UL, 0xbe5e65ccUL, 0xe0000000UL, + 0x3fe2fc15UL, 0x855a7390UL, 0xbe506438UL, 0x40000000UL, 0x3fe2f00bUL, + 0xa2898287UL, 0x3e3d22a2UL, 0xe0000000UL, 0x3fe2e403UL, 0x8b56f66fUL, + 0xbe5aa5fdUL, 0x80000000UL, 0x3fe2d7ffUL, 0x52db119aUL, 0x3e3a2e3dUL, + 0x60000000UL, 0x3fe2cbfeUL, 0xe2ddd4c0UL, 0xbe586469UL, 0x40000000UL, + 0x3fe2c000UL, 0x6b01bf10UL, 0x3e352b9dUL, 0x40000000UL, 0x3fe2b405UL, + 0xb07a1cdfUL, 0x3e5c5cdaUL, 0x80000000UL, 0x3fe2a80dUL, 0xc7b5f868UL, + 0xbe5668b3UL, 0xc0000000UL, 0x3fe29c18UL, 0x185edf62UL, 0xbe563d66UL, + 0x00000000UL, 0x3fe29027UL, 0xf729e1ccUL, 0x3e59a9a0UL, 0x80000000UL, + 0x3fe28438UL, 0x6433c727UL, 0xbe43cc89UL, 0x00000000UL, 0x3fe2784dUL, + 0x41782631UL, 0xbe30750cUL, 0xa0000000UL, 0x3fe26c64UL, 0x914911b7UL, + 0xbe58290eUL, 0x40000000UL, 0x3fe2607fUL, 0x3dcc73e1UL, 0xbe4269cdUL, + 0x00000000UL, 0x3fe2549dUL, 0x2751bf70UL, 0xbe5a6998UL, 0xc0000000UL, + 0x3fe248bdUL, 0x4248b9fbUL, 0xbe4ddb00UL, 0x80000000UL, 0x3fe23ce1UL, + 0xf35cf82fUL, 0x3e561b71UL, 0x60000000UL, 0x3fe23108UL, 0x8e481a2dUL, + 0x3e518fb9UL, 0x60000000UL, 0x3fe22532UL, 0x5ab96edcUL, 0xbe5fafc5UL, + 0x40000000UL, 0x3fe2195fUL, 0x80943911UL, 0xbe07f819UL, 0x40000000UL, + 0x3fe20d8fUL, 0x386f2d6cUL, 0xbe54ba8bUL, 0x40000000UL, 0x3fe201c2UL, + 0xf29664acUL, 0xbe5eb815UL, 0x20000000UL, 0x3fe1f5f8UL, 0x64f03390UL, + 0x3e5e320cUL, 0x20000000UL, 0x3fe1ea31UL, 0x747ff696UL, 0x3e5ef0a5UL, + 0x40000000UL, 0x3fe1de6dUL, 0x3e9ceb51UL, 0xbe5f8d27UL, 0x20000000UL, + 0x3fe1d2acUL, 0x4ae0b55eUL, 0x3e5faa21UL, 0x20000000UL, 0x3fe1c6eeUL, + 0x28569a5eUL, 0x3e598a4fUL, 0x20000000UL, 0x3fe1bb33UL, 0x54b33e07UL, + 0x3e46130aUL, 0x20000000UL, 0x3fe1af7bUL, 0x024f1078UL, 0xbe4dbf93UL, + 0x00000000UL, 0x3fe1a3c6UL, 0xb0783bfaUL, 0x3e419248UL, 0xe0000000UL, + 0x3fe19813UL, 0x2f02b836UL, 0x3e4e02b7UL, 0xc0000000UL, 0x3fe18c64UL, + 0x28dec9d4UL, 0x3e09064fUL, 0x80000000UL, 0x3fe180b8UL, 0x45cbf406UL, + 0x3e5b1f46UL, 0x40000000UL, 0x3fe1750fUL, 0x03d9964cUL, 0x3e5b0a79UL, + 0x00000000UL, 0x3fe16969UL, 0x8b5b882bUL, 0xbe238086UL, 0xa0000000UL, + 0x3fe15dc5UL, 0x73bad6f8UL, 0xbdf1fca4UL, 0x20000000UL, 0x3fe15225UL, + 0x5385769cUL, 0x3e5e8d76UL, 0xa0000000UL, 0x3fe14687UL, 0x1676dc6bUL, + 0x3e571d08UL, 0x20000000UL, 0x3fe13aedUL, 0xa8c41c7fUL, 0xbe598a25UL, + 0x60000000UL, 0x3fe12f55UL, 0xc4e1aaf0UL, 0x3e435277UL, 0xa0000000UL, + 0x3fe123c0UL, 0x403638e1UL, 0xbe21aa7cUL, 0xc0000000UL, 0x3fe1182eUL, + 0x557a092bUL, 0xbdd0116bUL, 0xc0000000UL, 0x3fe10c9fUL, 0x7d779f66UL, + 0x3e4a61baUL, 0xc0000000UL, 0x3fe10113UL, 0x2b09c645UL, 0xbe5d586eUL, + 0x20000000UL, 0x3fe0ea04UL, 0xea2cad46UL, 0x3e5aa97cUL, 0x20000000UL, + 0x3fe0d300UL, 0x23190e54UL, 0x3e50f1a7UL, 0xa0000000UL, 0x3fe0bc07UL, + 0x1379a5a6UL, 0xbe51619dUL, 0x60000000UL, 0x3fe0a51aUL, 0x926a3d4aUL, + 0x3e5cf019UL, 0xa0000000UL, 0x3fe08e38UL, 0xa8c24358UL, 0x3e35241eUL, + 0x20000000UL, 0x3fe07762UL, 0x24317e7aUL, 0x3e512cfaUL, 0x00000000UL, + 0x3fe06097UL, 0xfd9cf274UL, 0xbe55bef3UL, 0x00000000UL, 0x3fe049d7UL, + 0x3689b49dUL, 0xbe36d26dUL, 0x40000000UL, 0x3fe03322UL, 0xf72ef6c4UL, + 0xbe54cd08UL, 0xa0000000UL, 0x3fe01c78UL, 0x23702d2dUL, 0xbe5900bfUL, + 0x00000000UL, 0x3fe005daUL, 0x3f59c14cUL, 0x3e57d80bUL, 0x40000000UL, + 0x3fdfde8dUL, 0xad67766dUL, 0xbe57fad4UL, 0x40000000UL, 0x3fdfb17cUL, + 0x644f4ae7UL, 0x3e1ee43bUL, 0x40000000UL, 0x3fdf8481UL, 0x903234d2UL, + 0x3e501a86UL, 0x40000000UL, 0x3fdf579cUL, 0xafe9e509UL, 0xbe267c3eUL, + 0x00000000UL, 0x3fdf2acdUL, 0xb7dfda0bUL, 0xbe48149bUL, 0x40000000UL, + 0x3fdefe13UL, 0x3b94305eUL, 0x3e5f4ea7UL, 0x80000000UL, 0x3fded16fUL, + 0x5d95da61UL, 0xbe55c198UL, 0x00000000UL, 0x3fdea4e1UL, 0x406960c9UL, + 0xbdd99a19UL, 0x00000000UL, 0x3fde7868UL, 0xd22f3539UL, 0x3e470c78UL, + 0x80000000UL, 0x3fde4c04UL, 0x83eec535UL, 0xbe3e1232UL, 0x40000000UL, + 0x3fde1fb6UL, 0x3dfbffcbUL, 0xbe4b7d71UL, 0x40000000UL, 0x3fddf37dUL, + 0x7e1be4e0UL, 0xbe5b8f8fUL, 0x40000000UL, 0x3fddc759UL, 0x46dae887UL, + 0xbe350458UL, 0x80000000UL, 0x3fdd9b4aUL, 0xed6ecc49UL, 0xbe5f0045UL, + 0x80000000UL, 0x3fdd6f50UL, 0x2e9e883cUL, 0x3e2915daUL, 0x80000000UL, + 0x3fdd436bUL, 0xf0bccb32UL, 0x3e4a68c9UL, 0x80000000UL, 0x3fdd179bUL, + 0x9bbfc779UL, 0xbe54a26aUL, 0x00000000UL, 0x3fdcebe0UL, 0x7cea33abUL, + 0x3e43c6b7UL, 0x40000000UL, 0x3fdcc039UL, 0xe740fd06UL, 0x3e5526c2UL, + 0x40000000UL, 0x3fdc94a7UL, 0x9eadeb1aUL, 0xbe396d8dUL, 0xc0000000UL, + 0x3fdc6929UL, 0xf0a8f95aUL, 0xbe5c0ab2UL, 0x80000000UL, 0x3fdc3dc0UL, + 0x6ee2693bUL, 0x3e0992e6UL, 0xc0000000UL, 0x3fdc126bUL, 0x5ac6b581UL, + 0xbe2834b6UL, 0x40000000UL, 0x3fdbe72bUL, 0x8cc226ffUL, 0x3e3596a6UL, + 0x00000000UL, 0x3fdbbbffUL, 0xf92a74bbUL, 0x3e3c5813UL, 0x00000000UL, + 0x3fdb90e7UL, 0x479664c0UL, 0xbe50d644UL, 0x00000000UL, 0x3fdb65e3UL, + 0x5004975bUL, 0xbe55258fUL, 0x00000000UL, 0x3fdb3af3UL, 0xe4b23194UL, + 0xbe588407UL, 0xc0000000UL, 0x3fdb1016UL, 0xe65d4d0aUL, 0x3e527c26UL, + 0x80000000UL, 0x3fdae54eUL, 0x814fddd6UL, 0x3e5962a2UL, 0x40000000UL, + 0x3fdaba9aUL, 0xe19d0913UL, 0xbe562f4eUL, 0x80000000UL, 0x3fda8ff9UL, + 0x43cfd006UL, 0xbe4cfdebUL, 0x40000000UL, 0x3fda656cUL, 0x686f0a4eUL, + 0x3e5e47a8UL, 0xc0000000UL, 0x3fda3af2UL, 0x7200d410UL, 0x3e5e1199UL, + 0xc0000000UL, 0x3fda108cUL, 0xabd2266eUL, 0x3e5ee4d1UL, 0x40000000UL, + 0x3fd9e63aUL, 0x396f8f2cUL, 0x3e4dbffbUL, 0x00000000UL, 0x3fd9bbfbUL, + 0xe32b25ddUL, 0x3e5c3a54UL, 0x40000000UL, 0x3fd991cfUL, 0x431e4035UL, + 0xbe457925UL, 0x80000000UL, 0x3fd967b6UL, 0x7bed3dd3UL, 0x3e40c61dUL, + 0x00000000UL, 0x3fd93db1UL, 0xd7449365UL, 0x3e306419UL, 0x80000000UL, + 0x3fd913beUL, 0x1746e791UL, 0x3e56fcfcUL, 0x40000000UL, 0x3fd8e9dfUL, + 0xf3a9028bUL, 0xbe5041b9UL, 0xc0000000UL, 0x3fd8c012UL, 0x56840c50UL, + 0xbe26e20aUL, 0x40000000UL, 0x3fd89659UL, 0x19763102UL, 0xbe51f466UL, + 0x80000000UL, 0x3fd86cb2UL, 0x7032de7cUL, 0xbe4d298aUL, 0x80000000UL, + 0x3fd8431eUL, 0xdeb39fabUL, 0xbe4361ebUL, 0x40000000UL, 0x3fd8199dUL, + 0x5d01cbe0UL, 0xbe5425b3UL, 0x80000000UL, 0x3fd7f02eUL, 0x3ce99aa9UL, + 0x3e146fa8UL, 0x80000000UL, 0x3fd7c6d2UL, 0xd1a262b9UL, 0xbe5a1a69UL, + 0xc0000000UL, 0x3fd79d88UL, 0x8606c236UL, 0x3e423a08UL, 0x80000000UL, + 0x3fd77451UL, 0x8fd1e1b7UL, 0x3e5a6a63UL, 0xc0000000UL, 0x3fd74b2cUL, + 0xe491456aUL, 0x3e42c1caUL, 0x40000000UL, 0x3fd7221aUL, 0x4499a6d7UL, + 0x3e36a69aUL, 0x00000000UL, 0x3fd6f91aUL, 0x5237df94UL, 0xbe0f8f02UL, + 0x00000000UL, 0x3fd6d02cUL, 0xb6482c6eUL, 0xbe5abcf7UL, 0x00000000UL, + 0x3fd6a750UL, 0x1919fd61UL, 0xbe57ade2UL, 0x00000000UL, 0x3fd67e86UL, + 0xaa7a994dUL, 0xbe3f3fbdUL, 0x00000000UL, 0x3fd655ceUL, 0x67db014cUL, + 0x3e33c550UL, 0x00000000UL, 0x3fd62d28UL, 0xa82856b7UL, 0xbe1409d1UL, + 0xc0000000UL, 0x3fd60493UL, 0x1e6a300dUL, 0x3e55d899UL, 0x80000000UL, + 0x3fd5dc11UL, 0x1222bd5cUL, 0xbe35bfc0UL, 0xc0000000UL, 0x3fd5b3a0UL, + 0x6e8dc2d3UL, 0x3e5d4d79UL, 0x00000000UL, 0x3fd58b42UL, 0xe0e4ace6UL, + 0xbe517303UL, 0x80000000UL, 0x3fd562f4UL, 0xb306e0a8UL, 0x3e5edf0fUL, + 0xc0000000UL, 0x3fd53ab8UL, 0x6574bc54UL, 0x3e5ee859UL, 0x80000000UL, + 0x3fd5128eUL, 0xea902207UL, 0x3e5f6188UL, 0xc0000000UL, 0x3fd4ea75UL, + 0x9f911d79UL, 0x3e511735UL, 0x80000000UL, 0x3fd4c26eUL, 0xf9c77397UL, + 0xbe5b1643UL, 0x40000000UL, 0x3fd49a78UL, 0x15fc9258UL, 0x3e479a37UL, + 0x80000000UL, 0x3fd47293UL, 0xd5a04dd9UL, 0xbe426e56UL, 0xc0000000UL, + 0x3fd44abfUL, 0xe04042f5UL, 0x3e56f7c6UL, 0x40000000UL, 0x3fd422fdUL, + 0x1d8bf2c8UL, 0x3e5d8810UL, 0x00000000UL, 0x3fd3fb4cUL, 0x88a8ddeeUL, + 0xbe311454UL, 0xc0000000UL, 0x3fd3d3abUL, 0x3e3b5e47UL, 0xbe5d1b72UL, + 0x40000000UL, 0x3fd3ac1cUL, 0xc2ab5d59UL, 0x3e31b02bUL, 0xc0000000UL, + 0x3fd3849dUL, 0xd4e34b9eUL, 0x3e51cb2fUL, 0x40000000UL, 0x3fd35d30UL, + 0x177204fbUL, 0xbe2b8cd7UL, 0x80000000UL, 0x3fd335d3UL, 0xfcd38c82UL, + 0xbe4356e1UL, 0x80000000UL, 0x3fd30e87UL, 0x64f54accUL, 0xbe4e6224UL, + 0x00000000UL, 0x3fd2e74cUL, 0xaa7975d9UL, 0x3e5dc0feUL, 0x80000000UL, + 0x3fd2c021UL, 0x516dab3fUL, 0xbe50ffa3UL, 0x40000000UL, 0x3fd29907UL, + 0x2bfb7313UL, 0x3e5674a2UL, 0xc0000000UL, 0x3fd271fdUL, 0x0549fc99UL, + 0x3e385d29UL, 0xc0000000UL, 0x3fd24b04UL, 0x55b63073UL, 0xbe500c6dUL, + 0x00000000UL, 0x3fd2241cUL, 0x3f91953aUL, 0x3e389977UL, 0xc0000000UL, + 0x3fd1fd43UL, 0xa1543f71UL, 0xbe3487abUL, 0xc0000000UL, 0x3fd1d67bUL, + 0x4ec8867cUL, 0x3df6a2dcUL, 0x00000000UL, 0x3fd1afc4UL, 0x4328e3bbUL, + 0x3e41d9c0UL, 0x80000000UL, 0x3fd1891cUL, 0x2e1cda84UL, 0x3e3bdd87UL, + 0x40000000UL, 0x3fd16285UL, 0x4b5331aeUL, 0xbe53128eUL, 0x00000000UL, + 0x3fd13bfeUL, 0xb9aec164UL, 0xbe52ac98UL, 0xc0000000UL, 0x3fd11586UL, + 0xd91e1316UL, 0xbe350630UL, 0x80000000UL, 0x3fd0ef1fUL, 0x7cacc12cUL, + 0x3e3f5219UL, 0x40000000UL, 0x3fd0c8c8UL, 0xbce277b7UL, 0x3e3d30c0UL, + 0x00000000UL, 0x3fd0a281UL, 0x2a63447dUL, 0xbe541377UL, 0x80000000UL, + 0x3fd07c49UL, 0xfac483b5UL, 0xbe5772ecUL, 0xc0000000UL, 0x3fd05621UL, + 0x36b8a570UL, 0xbe4fd4bdUL, 0xc0000000UL, 0x3fd03009UL, 0xbae505f7UL, + 0xbe450388UL, 0x80000000UL, 0x3fd00a01UL, 0x3e35aeadUL, 0xbe5430fcUL, + 0x80000000UL, 0x3fcfc811UL, 0x707475acUL, 0x3e38806eUL, 0x80000000UL, + 0x3fcf7c3fUL, 0xc91817fcUL, 0xbe40cceaUL, 0x80000000UL, 0x3fcf308cUL, + 0xae05d5e9UL, 0xbe4919b8UL, 0x80000000UL, 0x3fcee4f8UL, 0xae6cc9e6UL, + 0xbe530b94UL, 0x00000000UL, 0x3fce9983UL, 0x1efe3e8eUL, 0x3e57747eUL, + 0x00000000UL, 0x3fce4e2dUL, 0xda78d9bfUL, 0xbe59a608UL, 0x00000000UL, + 0x3fce02f5UL, 0x8abe2c2eUL, 0x3e4a35adUL, 0x00000000UL, 0x3fcdb7dcUL, + 0x1495450dUL, 0xbe0872ccUL, 0x80000000UL, 0x3fcd6ce1UL, 0x86ee0ba0UL, + 0xbe4f59a0UL, 0x00000000UL, 0x3fcd2205UL, 0xe81ca888UL, 0x3e5402c3UL, + 0x00000000UL, 0x3fccd747UL, 0x3b4424b9UL, 0x3e5dfdc3UL, 0x80000000UL, + 0x3fcc8ca7UL, 0xd305b56cUL, 0x3e202da6UL, 0x00000000UL, 0x3fcc4226UL, + 0x399a6910UL, 0xbe482a1cUL, 0x80000000UL, 0x3fcbf7c2UL, 0x747f7938UL, + 0xbe587372UL, 0x80000000UL, 0x3fcbad7cUL, 0x6fc246a0UL, 0x3e50d83dUL, + 0x00000000UL, 0x3fcb6355UL, 0xee9e9be5UL, 0xbe5c35bdUL, 0x80000000UL, + 0x3fcb194aUL, 0x8416c0bcUL, 0x3e546d4fUL, 0x00000000UL, 0x3fcacf5eUL, + 0x49f7f08fUL, 0x3e56da76UL, 0x00000000UL, 0x3fca858fUL, 0x5dc30de2UL, + 0x3e5f390cUL, 0x00000000UL, 0x3fca3bdeUL, 0x950583b6UL, 0xbe5e4169UL, + 0x80000000UL, 0x3fc9f249UL, 0x33631553UL, 0x3e52aeb1UL, 0x00000000UL, + 0x3fc9a8d3UL, 0xde8795a6UL, 0xbe59a504UL, 0x00000000UL, 0x3fc95f79UL, + 0x076bf41eUL, 0x3e5122feUL, 0x80000000UL, 0x3fc9163cUL, 0x2914c8e7UL, + 0x3e3dd064UL, 0x00000000UL, 0x3fc8cd1dUL, 0x3a30eca3UL, 0xbe21b4aaUL, + 0x80000000UL, 0x3fc8841aUL, 0xb2a96650UL, 0xbe575444UL, 0x80000000UL, + 0x3fc83b34UL, 0x2376c0cbUL, 0xbe2a74c7UL, 0x80000000UL, 0x3fc7f26bUL, + 0xd8a0b653UL, 0xbe5181b6UL, 0x00000000UL, 0x3fc7a9bfUL, 0x32257882UL, + 0xbe4a78b4UL, 0x00000000UL, 0x3fc7612fUL, 0x1eee8bd9UL, 0xbe1bfe9dUL, + 0x80000000UL, 0x3fc718bbUL, 0x0c603cc4UL, 0x3e36fdc9UL, 0x80000000UL, + 0x3fc6d064UL, 0x3728b8cfUL, 0xbe1e542eUL, 0x80000000UL, 0x3fc68829UL, + 0xc79a4067UL, 0x3e5c380fUL, 0x00000000UL, 0x3fc6400bUL, 0xf69eac69UL, + 0x3e550a84UL, 0x80000000UL, 0x3fc5f808UL, 0xb7a780a4UL, 0x3e5d9224UL, + 0x80000000UL, 0x3fc5b022UL, 0xad9dfb1eUL, 0xbe55242fUL, 0x00000000UL, + 0x3fc56858UL, 0x659b18beUL, 0xbe4bfda3UL, 0x80000000UL, 0x3fc520a9UL, + 0x66ee3631UL, 0xbe57d769UL, 0x80000000UL, 0x3fc4d916UL, 0x1ec62819UL, + 0x3e2427f7UL, 0x80000000UL, 0x3fc4919fUL, 0xdec25369UL, 0xbe435431UL, + 0x00000000UL, 0x3fc44a44UL, 0xa8acfc4bUL, 0xbe3c62e8UL, 0x00000000UL, + 0x3fc40304UL, 0xcf1d3eabUL, 0xbdfba29fUL, 0x80000000UL, 0x3fc3bbdfUL, + 0x79aba3eaUL, 0xbdf1b7c8UL, 0x80000000UL, 0x3fc374d6UL, 0xb8d186daUL, + 0xbe5130cfUL, 0x80000000UL, 0x3fc32de8UL, 0x9d74f152UL, 0x3e2285b6UL, + 0x00000000UL, 0x3fc2e716UL, 0x50ae7ca9UL, 0xbe503920UL, 0x80000000UL, + 0x3fc2a05eUL, 0x6caed92eUL, 0xbe533924UL, 0x00000000UL, 0x3fc259c2UL, + 0x9cb5034eUL, 0xbe510e31UL, 0x80000000UL, 0x3fc21340UL, 0x12c4d378UL, + 0xbe540b43UL, 0x80000000UL, 0x3fc1ccd9UL, 0xcc418706UL, 0x3e59887aUL, + 0x00000000UL, 0x3fc1868eUL, 0x921f4106UL, 0xbe528e67UL, 0x80000000UL, + 0x3fc1405cUL, 0x3969441eUL, 0x3e5d8051UL, 0x00000000UL, 0x3fc0fa46UL, + 0xd941ef5bUL, 0x3e5f9079UL, 0x80000000UL, 0x3fc0b44aUL, 0x5a3e81b2UL, + 0xbe567691UL, 0x00000000UL, 0x3fc06e69UL, 0x9d66afe7UL, 0xbe4d43fbUL, + 0x00000000UL, 0x3fc028a2UL, 0x0a92a162UL, 0xbe52f394UL, 0x00000000UL, + 0x3fbfc5eaUL, 0x209897e5UL, 0x3e529e37UL, 0x00000000UL, 0x3fbf3ac5UL, + 0x8458bd7bUL, 0x3e582831UL, 0x00000000UL, 0x3fbeafd5UL, 0xb8d8b4b8UL, + 0xbe486b4aUL, 0x00000000UL, 0x3fbe2518UL, 0xe0a3b7b6UL, 0x3e5bafd2UL, + 0x00000000UL, 0x3fbd9a90UL, 0x2bf2710eUL, 0x3e383b2bUL, 0x00000000UL, + 0x3fbd103cUL, 0x73eb6ab7UL, 0xbe56d78dUL, 0x00000000UL, 0x3fbc861bUL, + 0x32ceaff5UL, 0xbe32dc5aUL, 0x00000000UL, 0x3fbbfc2eUL, 0xbee04cb7UL, + 0xbe4a71a4UL, 0x00000000UL, 0x3fbb7274UL, 0x35ae9577UL, 0x3e38142fUL, + 0x00000000UL, 0x3fbae8eeUL, 0xcbaddab4UL, 0xbe5490f0UL, 0x00000000UL, + 0x3fba5f9aUL, 0x95ce1114UL, 0x3e597c71UL, 0x00000000UL, 0x3fb9d67aUL, + 0x6d7c0f78UL, 0x3e3abc2dUL, 0x00000000UL, 0x3fb94d8dUL, 0x2841a782UL, + 0xbe566cbcUL, 0x00000000UL, 0x3fb8c4d2UL, 0x6ed429c6UL, 0xbe3cfff9UL, + 0x00000000UL, 0x3fb83c4aUL, 0xe4a49fbbUL, 0xbe552964UL, 0x00000000UL, + 0x3fb7b3f4UL, 0x2193d81eUL, 0xbe42fa72UL, 0x00000000UL, 0x3fb72bd0UL, + 0xdd70c122UL, 0x3e527a8cUL, 0x00000000UL, 0x3fb6a3dfUL, 0x03108a54UL, + 0xbe450393UL, 0x00000000UL, 0x3fb61c1fUL, 0x30ff7954UL, 0x3e565840UL, + 0x00000000UL, 0x3fb59492UL, 0xdedd460cUL, 0xbe5422b5UL, 0x00000000UL, + 0x3fb50d36UL, 0x950f9f45UL, 0xbe5313f6UL, 0x00000000UL, 0x3fb4860bUL, + 0x582cdcb1UL, 0x3e506d39UL, 0x00000000UL, 0x3fb3ff12UL, 0x7216d3a6UL, + 0x3e4aa719UL, 0x00000000UL, 0x3fb3784aUL, 0x57a423fdUL, 0x3e5a9b9fUL, + 0x00000000UL, 0x3fb2f1b4UL, 0x7a138b41UL, 0xbe50b418UL, 0x00000000UL, + 0x3fb26b4eUL, 0x2fbfd7eaUL, 0x3e23a53eUL, 0x00000000UL, 0x3fb1e519UL, + 0x18913ccbUL, 0x3e465fc1UL, 0x00000000UL, 0x3fb15f15UL, 0x7ea24e21UL, + 0x3e042843UL, 0x00000000UL, 0x3fb0d941UL, 0x7c6d9c77UL, 0x3e59f61eUL, + 0x00000000UL, 0x3fb0539eUL, 0x114efd44UL, 0x3e4ccab7UL, 0x00000000UL, + 0x3faf9c56UL, 0x1777f657UL, 0x3e552f65UL, 0x00000000UL, 0x3fae91d2UL, + 0xc317b86aUL, 0xbe5a61e0UL, 0x00000000UL, 0x3fad87acUL, 0xb7664efbUL, + 0xbe41f64eUL, 0x00000000UL, 0x3fac7de6UL, 0x5d3d03a9UL, 0x3e0807a0UL, + 0x00000000UL, 0x3fab7480UL, 0x743c38ebUL, 0xbe3726e1UL, 0x00000000UL, + 0x3faa6b78UL, 0x06a253f1UL, 0x3e5ad636UL, 0x00000000UL, 0x3fa962d0UL, + 0xa35f541bUL, 0x3e5a187aUL, 0x00000000UL, 0x3fa85a88UL, 0x4b86e446UL, + 0xbe508150UL, 0x00000000UL, 0x3fa7529cUL, 0x2589cacfUL, 0x3e52938aUL, + 0x00000000UL, 0x3fa64b10UL, 0xaf6b11f2UL, 0xbe3454cdUL, 0x00000000UL, + 0x3fa543e2UL, 0x97506fefUL, 0xbe5fdec5UL, 0x00000000UL, 0x3fa43d10UL, + 0xe75f7dd9UL, 0xbe388dd3UL, 0x00000000UL, 0x3fa3369cUL, 0xa4139632UL, + 0xbdea5177UL, 0x00000000UL, 0x3fa23086UL, 0x352d6f1eUL, 0xbe565ad6UL, + 0x00000000UL, 0x3fa12accUL, 0x77449eb7UL, 0xbe50d5c7UL, 0x00000000UL, + 0x3fa0256eUL, 0x7478da78UL, 0x3e404724UL, 0x00000000UL, 0x3f9e40dcUL, + 0xf59cef7fUL, 0xbe539d0aUL, 0x00000000UL, 0x3f9c3790UL, 0x1511d43cUL, + 0x3e53c2c8UL, 0x00000000UL, 0x3f9a2f00UL, 0x9b8bff3cUL, 0xbe43b3e1UL, + 0x00000000UL, 0x3f982724UL, 0xad1e22a5UL, 0x3e46f0bdUL, 0x00000000UL, + 0x3f962000UL, 0x130d9356UL, 0x3e475ba0UL, 0x00000000UL, 0x3f941994UL, + 0x8f86f883UL, 0xbe513d0bUL, 0x00000000UL, 0x3f9213dcUL, 0x914d0dc8UL, + 0xbe534335UL, 0x00000000UL, 0x3f900ed8UL, 0x2d73e5e7UL, 0xbe22ba75UL, + 0x00000000UL, 0x3f8c1510UL, 0xc5b7d70eUL, 0x3e599c5dUL, 0x00000000UL, + 0x3f880de0UL, 0x8a27857eUL, 0xbe3d28c8UL, 0x00000000UL, 0x3f840810UL, + 0xda767328UL, 0x3e531b3dUL, 0x00000000UL, 0x3f8003b0UL, 0x77bacaf3UL, + 0xbe5f04e3UL, 0x00000000UL, 0x3f780150UL, 0xdf4b0720UL, 0x3e5a8bffUL, + 0x00000000UL, 0x3f6ffc40UL, 0x34c48e71UL, 0xbe3fcd99UL, 0x00000000UL, + 0x3f5ff6c0UL, 0x1ad218afUL, 0xbe4c78a7UL, 0x00000000UL, 0x00000000UL, + 0x00000000UL, 0x80000000UL +}; + +ALIGNED_(8) juint _log2_pow[] = +{ + 0xfefa39efUL, 0x3fe62e42UL, 0xfefa39efUL, 0xbfe62e42UL +}; + +//registers, +// input: xmm0, xmm1 +// scratch: xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7 +// rax, rdx, rcx, r8, r11 + +// Code generated by Intel C compiler for LIBM library + +void MacroAssembler::fast_pow(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xmm2, XMMRegister xmm3, XMMRegister xmm4, XMMRegister xmm5, XMMRegister xmm6, XMMRegister xmm7, Register eax, Register ecx, Register edx, Register tmp1, Register tmp2, Register tmp3, Register tmp4) { + Label L_2TAG_PACKET_0_0_2, L_2TAG_PACKET_1_0_2, L_2TAG_PACKET_2_0_2, L_2TAG_PACKET_3_0_2; + Label L_2TAG_PACKET_4_0_2, L_2TAG_PACKET_5_0_2, L_2TAG_PACKET_6_0_2, L_2TAG_PACKET_7_0_2; + Label L_2TAG_PACKET_8_0_2, L_2TAG_PACKET_9_0_2, L_2TAG_PACKET_10_0_2, L_2TAG_PACKET_11_0_2; + Label L_2TAG_PACKET_12_0_2, L_2TAG_PACKET_13_0_2, L_2TAG_PACKET_14_0_2, L_2TAG_PACKET_15_0_2; + Label L_2TAG_PACKET_16_0_2, L_2TAG_PACKET_17_0_2, L_2TAG_PACKET_18_0_2, L_2TAG_PACKET_19_0_2; + Label L_2TAG_PACKET_20_0_2, L_2TAG_PACKET_21_0_2, L_2TAG_PACKET_22_0_2, L_2TAG_PACKET_23_0_2; + Label L_2TAG_PACKET_24_0_2, L_2TAG_PACKET_25_0_2, L_2TAG_PACKET_26_0_2, L_2TAG_PACKET_27_0_2; + Label L_2TAG_PACKET_28_0_2, L_2TAG_PACKET_29_0_2, L_2TAG_PACKET_30_0_2, L_2TAG_PACKET_31_0_2; + Label L_2TAG_PACKET_32_0_2, L_2TAG_PACKET_33_0_2, L_2TAG_PACKET_34_0_2, L_2TAG_PACKET_35_0_2; + Label L_2TAG_PACKET_36_0_2, L_2TAG_PACKET_37_0_2, L_2TAG_PACKET_38_0_2, L_2TAG_PACKET_39_0_2; + Label L_2TAG_PACKET_40_0_2, L_2TAG_PACKET_41_0_2, L_2TAG_PACKET_42_0_2, L_2TAG_PACKET_43_0_2; + Label L_2TAG_PACKET_44_0_2, L_2TAG_PACKET_45_0_2, L_2TAG_PACKET_46_0_2, L_2TAG_PACKET_47_0_2; + Label L_2TAG_PACKET_48_0_2, L_2TAG_PACKET_49_0_2, L_2TAG_PACKET_50_0_2, L_2TAG_PACKET_51_0_2; + Label L_2TAG_PACKET_52_0_2, L_2TAG_PACKET_53_0_2, L_2TAG_PACKET_54_0_2, L_2TAG_PACKET_55_0_2; + Label L_2TAG_PACKET_56_0_2; + Label B1_2, B1_3, B1_5, start; + + assert_different_registers(tmp1, tmp2, eax, ecx, edx); + jmp(start); + address HIGHSIGMASK = (address)_HIGHSIGMASK; + address LOG2_E = (address)_LOG2_E; + address coeff = (address)_coeff_pow; + address L_tbl = (address)_L_tbl_pow; + address HIGHMASK_Y = (address)_HIGHMASK_Y; + address T_exp = (address)_T_exp; + address e_coeff = (address)_e_coeff; + address coeff_h = (address)_coeff_h; + address HIGHMASK_LOG_X = (address)_HIGHMASK_LOG_X; + address HALFMASK = (address)_HALFMASK; + address log2 = (address)_log2_pow; + + + bind(start); + subq(rsp, 40); + movsd(Address(rsp, 8), xmm0); + movsd(Address(rsp, 16), xmm1); + + bind(B1_2); + pextrw(eax, xmm0, 3); + xorpd(xmm2, xmm2); + mov64(tmp2, 0x3ff0000000000000); + movdq(xmm2, tmp2); + movl(tmp1, 1069088768); + movdq(xmm7, tmp1); + xorpd(xmm1, xmm1); + mov64(tmp3, 0x77f0000000000000); + movdq(xmm1, tmp3); + movdqu(xmm3, xmm0); + movl(edx, 32752); + andl(edx, eax); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + por(xmm0, xmm2); + movdqu(xmm6, ExternalAddress(HIGHSIGMASK)); //0x00000000UL, 0xfffff800UL, 0x00000000UL, 0xfffff800UL + psrlq(xmm0, 27); + movq(xmm2, ExternalAddress(LOG2_E)); //0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL + psrld(xmm0, 2); + addl(ecx, 16); + bsrl(ecx, ecx); + rcpps(xmm0, xmm0); + psllq(xmm3, 12); + movl(tmp4, 8192); + movdq(xmm4, tmp4); + psrlq(xmm3, 12); + subl(eax, 16); + cmpl(eax, 32736); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_0_0_2); + movl(tmp1, 0); + + bind(L_2TAG_PACKET_1_0_2); + mulss(xmm0, xmm7); + movl(edx, -1); + subl(ecx, 4); + shll(edx); + shlq(edx, 32); + movdq(xmm5, edx); + por(xmm3, xmm1); + subl(eax, 16351); + cmpl(eax, 1); + jcc(Assembler::belowEqual, L_2TAG_PACKET_2_0_2); + paddd(xmm0, xmm4); + pand(xmm5, xmm3); + movdl(edx, xmm0); + psllq(xmm0, 29); + + bind(L_2TAG_PACKET_3_0_2); + subsd(xmm3, xmm5); + pand(xmm0, xmm6); + subl(eax, 1); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + + bind(L_2TAG_PACKET_4_0_2); + mulsd(xmm3, xmm0); + movdqu(xmm1, ExternalAddress(coeff)); //0x6dc96112UL, 0xbf836578UL, 0xee241472UL, 0xbf9b0301UL + lea(tmp4, ExternalAddress(L_tbl)); + subsd(xmm5, xmm2); + movdqu(xmm4, ExternalAddress(16 + coeff)); //0x9f95985aUL, 0xbfb528dbUL, 0xb3841d2aUL, 0xbfd619b6UL + movl(ecx, eax); + sarl(eax, 31); + addl(ecx, eax); + xorl(eax, ecx); + addl(eax, 1); + bsrl(eax, eax); + unpcklpd(xmm5, xmm3); + movdqu(xmm6, ExternalAddress(32 + coeff)); //0x518775e3UL, 0x3f9004f2UL, 0xac8349bbUL, 0x3fa76c9bUL + addsd(xmm3, xmm5); + andl(edx, 16760832); + shrl(edx, 10); + addpd(xmm5, Address(tmp4, edx, Address::times_1, -3648)); + movdqu(xmm0, ExternalAddress(48 + coeff)); //0x486ececcUL, 0x3fc4635eUL, 0x161bb241UL, 0xbf5dabe1UL + pshufd(xmm2, xmm3, 68); + mulsd(xmm3, xmm3); + mulpd(xmm1, xmm2); + mulpd(xmm4, xmm2); + addsd(xmm5, xmm7); + mulsd(xmm2, xmm3); + addpd(xmm6, xmm1); + mulsd(xmm3, xmm3); + addpd(xmm0, xmm4); + movq(xmm1, Address(rsp, 16)); + movw(ecx, Address(rsp, 22)); + pshufd(xmm7, xmm5, 238); + movq(xmm4, ExternalAddress(HIGHMASK_Y)); //0x00000000UL, 0xfffffff8UL, 0x00000000UL, 0xffffffffUL + mulpd(xmm6, xmm2); + pshufd(xmm3, xmm3, 68); + mulpd(xmm0, xmm2); + shll(eax, 4); + subl(eax, 15872); + andl(ecx, 32752); + addl(eax, ecx); + mulpd(xmm3, xmm6); + cmpl(eax, 624); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_5_0_2); + xorpd(xmm6, xmm6); + movl(edx, 17080); + pinsrw(xmm6, edx, 3); + movdqu(xmm2, xmm1); + pand(xmm4, xmm1); + subsd(xmm1, xmm4); + mulsd(xmm4, xmm5); + addsd(xmm0, xmm7); + mulsd(xmm1, xmm5); + movdqu(xmm7, xmm6); + addsd(xmm6, xmm4); + lea(tmp4, ExternalAddress(T_exp)); + addpd(xmm3, xmm0); + movdl(edx, xmm6); + subsd(xmm6, xmm7); + pshufd(xmm0, xmm3, 238); + subsd(xmm4, xmm6); + addsd(xmm0, xmm3); + movl(ecx, edx); + andl(edx, 255); + addl(edx, edx); + movdqu(xmm5, Address(tmp4, edx, Address::times_8, 0)); + addsd(xmm4, xmm1); + mulsd(xmm2, xmm0); + movdqu(xmm7, ExternalAddress(e_coeff)); //0xe78a6731UL, 0x3f55d87fUL, 0xd704a0c0UL, 0x3fac6b08UL + movdqu(xmm3, ExternalAddress(16 + e_coeff)); //0x6fba4e77UL, 0x3f83b2abUL, 0xff82c58fUL, 0x3fcebfbdUL + shll(ecx, 12); + xorl(ecx, tmp1); + andl(rcx, -1048576); + movdq(xmm6, rcx); + addsd(xmm2, xmm4); + mov64(tmp2, 0x3fe62e42fefa39ef); + movdq(xmm1, tmp2); + pshufd(xmm0, xmm2, 68); + pshufd(xmm4, xmm2, 68); + mulsd(xmm1, xmm2); + pshufd(xmm6, xmm6, 17); + mulpd(xmm0, xmm0); + mulpd(xmm7, xmm4); + paddd(xmm5, xmm6); + mulsd(xmm1, xmm5); + pshufd(xmm6, xmm5, 238); + mulsd(xmm0, xmm0); + addpd(xmm3, xmm7); + addsd(xmm1, xmm6); + mulpd(xmm0, xmm3); + pshufd(xmm3, xmm0, 238); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + addsd(xmm0, xmm1); + addsd(xmm0, xmm3); + addsd(xmm0, xmm5); + jmp(B1_5); + + bind(L_2TAG_PACKET_0_0_2); + addl(eax, 16); + movl(edx, 32752); + andl(edx, eax); + cmpl(edx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_6_0_2); + testl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_7_0_2); + + bind(L_2TAG_PACKET_8_0_2); + movq(xmm0, Address(rsp, 8)); + movq(xmm3, Address(rsp, 8)); + movdl(edx, xmm3); + psrlq(xmm3, 32); + movdl(ecx, xmm3); + orl(edx, ecx); + cmpl(edx, 0); + jcc(Assembler::equal, L_2TAG_PACKET_9_0_2); + xorpd(xmm3, xmm3); + movl(eax, 18416); + pinsrw(xmm3, eax, 3); + mulsd(xmm0, xmm3); + xorpd(xmm2, xmm2); + movl(eax, 16368); + pinsrw(xmm2, eax, 3); + movdqu(xmm3, xmm0); + pextrw(eax, xmm0, 3); + por(xmm0, xmm2); + movl(ecx, 18416); + psrlq(xmm0, 27); + movq(xmm2, ExternalAddress(LOG2_E)); //0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL + psrld(xmm0, 2); + rcpps(xmm0, xmm0); + psllq(xmm3, 12); + movdqu(xmm6, ExternalAddress(HIGHSIGMASK)); //0x00000000UL, 0xfffff800UL, 0x00000000UL, 0xfffff800UL + psrlq(xmm3, 12); + mulss(xmm0, xmm7); + movl(edx, -1024); + movdl(xmm5, edx); + por(xmm3, xmm1); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + pand(xmm5, xmm3); + movl(tmp1, 0); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + andl(eax, 32752); + subl(eax, 18416); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + jmp(L_2TAG_PACKET_4_0_2); + + bind(L_2TAG_PACKET_10_0_2); + movq(xmm0, Address(rsp, 8)); + movq(xmm3, Address(rsp, 8)); + movdl(edx, xmm3); + psrlq(xmm3, 32); + movdl(ecx, xmm3); + orl(edx, ecx); + cmpl(edx, 0); + jcc(Assembler::equal, L_2TAG_PACKET_9_0_2); + xorpd(xmm3, xmm3); + movl(eax, 18416); + pinsrw(xmm3, eax, 3); + mulsd(xmm0, xmm3); + xorpd(xmm2, xmm2); + movl(eax, 16368); + pinsrw(xmm2, eax, 3); + movdqu(xmm3, xmm0); + pextrw(eax, xmm0, 3); + por(xmm0, xmm2); + movl(ecx, 18416); + psrlq(xmm0, 27); + movq(xmm2, ExternalAddress(LOG2_E)); //0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL + psrld(xmm0, 2); + rcpps(xmm0, xmm0); + psllq(xmm3, 12); + movdqu(xmm6, ExternalAddress(HIGHSIGMASK)); //0x00000000UL, 0xfffff800UL, 0x00000000UL, 0xfffff800UL + psrlq(xmm3, 12); + mulss(xmm0, xmm7); + movl(edx, -1024); + movdl(xmm5, edx); + por(xmm3, xmm1); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + pand(xmm5, xmm3); + movl(tmp1, INT_MIN); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + andl(eax, 32752); + subl(eax, 18416); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + jmp(L_2TAG_PACKET_4_0_2); + + bind(L_2TAG_PACKET_5_0_2); + cmpl(eax, 0); + jcc(Assembler::less, L_2TAG_PACKET_11_0_2); + cmpl(eax, 752); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_12_0_2); + addsd(xmm0, xmm7); + movq(xmm2, ExternalAddress(HALFMASK)); //0xf8000000UL, 0xffffffffUL, 0xf8000000UL, 0xffffffffUL + addpd(xmm3, xmm0); + xorpd(xmm6, xmm6); + movl(eax, 17080); + pinsrw(xmm6, eax, 3); + pshufd(xmm0, xmm3, 238); + addsd(xmm0, xmm3); + movdqu(xmm3, xmm5); + addsd(xmm5, xmm0); + movdqu(xmm4, xmm2); + subsd(xmm3, xmm5); + movdqu(xmm7, xmm5); + pand(xmm5, xmm2); + movdqu(xmm2, xmm1); + pand(xmm4, xmm1); + subsd(xmm7, xmm5); + addsd(xmm0, xmm3); + subsd(xmm1, xmm4); + mulsd(xmm4, xmm5); + addsd(xmm0, xmm7); + mulsd(xmm2, xmm0); + movdqu(xmm7, xmm6); + mulsd(xmm1, xmm5); + addsd(xmm6, xmm4); + movdl(eax, xmm6); + subsd(xmm6, xmm7); + lea(tmp4, ExternalAddress(T_exp)); + addsd(xmm2, xmm1); + movdqu(xmm7, ExternalAddress(e_coeff)); //0xe78a6731UL, 0x3f55d87fUL, 0xd704a0c0UL, 0x3fac6b08UL + movdqu(xmm3, ExternalAddress(16 + e_coeff)); //0x6fba4e77UL, 0x3f83b2abUL, 0xff82c58fUL, 0x3fcebfbdUL + subsd(xmm4, xmm6); + pextrw(edx, xmm6, 3); + movl(ecx, eax); + andl(eax, 255); + addl(eax, eax); + movdqu(xmm5, Address(tmp4, rax, Address::times_8, 0)); + addsd(xmm2, xmm4); + sarl(ecx, 8); + movl(eax, ecx); + sarl(ecx, 1); + subl(eax, ecx); + shll(ecx, 20); + xorl(ecx, tmp1); + movdl(xmm6, ecx); + movq(xmm1, ExternalAddress(32 + e_coeff)); //0xfefa39efUL, 0x3fe62e42UL, 0x00000000UL, 0x00000000UL + andl(edx, 32767); + cmpl(edx, 16529); + jcc(Assembler::above, L_2TAG_PACKET_12_0_2); + pshufd(xmm0, xmm2, 68); + pshufd(xmm4, xmm2, 68); + mulpd(xmm0, xmm0); + mulpd(xmm7, xmm4); + pshufd(xmm6, xmm6, 17); + mulsd(xmm1, xmm2); + mulsd(xmm0, xmm0); + paddd(xmm5, xmm6); + addpd(xmm3, xmm7); + mulsd(xmm1, xmm5); + pshufd(xmm6, xmm5, 238); + mulpd(xmm0, xmm3); + addsd(xmm1, xmm6); + pshufd(xmm3, xmm0, 238); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + shll(eax, 4); + xorpd(xmm4, xmm4); + addl(eax, 16368); + pinsrw(xmm4, eax, 3); + addsd(xmm0, xmm1); + addsd(xmm0, xmm3); + movdqu(xmm1, xmm0); + addsd(xmm0, xmm5); + mulsd(xmm0, xmm4); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_13_0_2); + cmpl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_14_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_6_0_2); + movq(xmm1, Address(rsp, 16)); + movq(xmm0, Address(rsp, 8)); + movdqu(xmm2, xmm0); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_15_0_2); + movdl(eax, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_16_0_2); + addsd(xmm0, xmm0); + jmp(B1_5); + + bind(L_2TAG_PACKET_16_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + movl(Address(rsp, 0), 29); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_18_0_2); + movq(xmm0, Address(rsp, 16)); + addpd(xmm0, xmm0); + jmp(B1_5); + + bind(L_2TAG_PACKET_15_0_2); + movdl(eax, xmm1); + movdqu(xmm2, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_19_0_2); + pextrw(eax, xmm2, 3); + andl(eax, 32752); + cmpl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_20_0_2); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_20_0_2); + pextrw(eax, xmm0, 3); + testl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_21_0_2); + testl(ecx, INT_MIN); + jcc(Assembler::notEqual, L_2TAG_PACKET_22_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_23_0_2); + movq(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_24_0_2); + testl(eax, 2); + jcc(Assembler::notEqual, L_2TAG_PACKET_25_0_2); + jmp(L_2TAG_PACKET_24_0_2); + + bind(L_2TAG_PACKET_21_0_2); + shrl(ecx, 20); + andl(ecx, 2047); + cmpl(ecx, 1075); + jcc(Assembler::above, L_2TAG_PACKET_24_0_2); + jcc(Assembler::equal, L_2TAG_PACKET_26_0_2); + cmpl(ecx, 1074); + jcc(Assembler::above, L_2TAG_PACKET_23_0_2); + cmpl(ecx, 1023); + jcc(Assembler::below, L_2TAG_PACKET_24_0_2); + movq(xmm1, Address(rsp, 16)); + movl(eax, 17208); + xorpd(xmm3, xmm3); + pinsrw(xmm3, eax, 3); + movdqu(xmm4, xmm3); + addsd(xmm3, xmm1); + subsd(xmm4, xmm3); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_24_0_2); + movdl(eax, xmm3); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_24_0_2); + + bind(L_2TAG_PACKET_25_0_2); + movq(xmm1, Address(rsp, 16)); + pextrw(eax, xmm1, 3); + andl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_27_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_27_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32768); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_24_0_2); + movq(xmm1, Address(rsp, 16)); + pextrw(eax, xmm1, 3); + andl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_22_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32752); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_26_0_2); + movq(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_24_0_2); + jmp(L_2TAG_PACKET_25_0_2); + + bind(L_2TAG_PACKET_28_0_2); + movdl(eax, xmm1); + psrlq(xmm1, 20); + movdl(edx, xmm1); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_29_0_2); + movq(xmm0, Address(rsp, 16)); + addsd(xmm0, xmm0); + jmp(B1_5); + + bind(L_2TAG_PACKET_29_0_2); + movq(xmm0, Address(rsp, 8)); + pextrw(eax, xmm0, 3); + cmpl(eax, 49136); + jcc(Assembler::notEqual, L_2TAG_PACKET_30_0_2); + movdl(ecx, xmm0); + psrlq(xmm0, 20); + movdl(edx, xmm0); + orl(ecx, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_30_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32760); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_30_0_2); + movq(xmm1, Address(rsp, 16)); + andl(eax, 32752); + subl(eax, 16368); + pextrw(edx, xmm1, 3); + xorpd(xmm0, xmm0); + xorl(eax, edx); + andl(eax, 32768); + jcc(Assembler::equal, L_2TAG_PACKET_31_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_31_0_2); + movl(ecx, 32752); + pinsrw(xmm0, ecx, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_32_0_2); + movdl(eax, xmm1); + cmpl(edx, 17184); + jcc(Assembler::above, L_2TAG_PACKET_33_0_2); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_34_0_2); + testl(eax, 2); + jcc(Assembler::equal, L_2TAG_PACKET_35_0_2); + jmp(L_2TAG_PACKET_36_0_2); + + bind(L_2TAG_PACKET_33_0_2); + testl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_35_0_2); + jmp(L_2TAG_PACKET_36_0_2); + + bind(L_2TAG_PACKET_7_0_2); + movq(xmm2, Address(rsp, 8)); + movdl(eax, xmm2); + psrlq(xmm2, 31); + movdl(ecx, xmm2); + orl(eax, ecx); + jcc(Assembler::equal, L_2TAG_PACKET_9_0_2); + movq(xmm1, Address(rsp, 16)); + pextrw(edx, xmm1, 3); + movdl(eax, xmm1); + movdqu(xmm2, xmm1); + psrlq(xmm2, 32); + movdl(ecx, xmm2); + addl(ecx, ecx); + orl(ecx, eax); + jcc(Assembler::equal, L_2TAG_PACKET_37_0_2); + andl(edx, 32752); + cmpl(edx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_28_0_2); + cmpl(edx, 17200); + jcc(Assembler::above, L_2TAG_PACKET_35_0_2); + cmpl(edx, 17184); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_32_0_2); + cmpl(edx, 16368); + jcc(Assembler::below, L_2TAG_PACKET_34_0_2); + movl(eax, 17208); + xorpd(xmm2, xmm2); + pinsrw(xmm2, eax, 3); + movdqu(xmm4, xmm2); + addsd(xmm2, xmm1); + subsd(xmm4, xmm2); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32767); + jcc(Assembler::notEqual, L_2TAG_PACKET_34_0_2); + movdl(eax, xmm2); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_35_0_2); + + bind(L_2TAG_PACKET_36_0_2); + xorpd(xmm1, xmm1); + movl(edx, 30704); + pinsrw(xmm1, edx, 3); + movq(xmm2, ExternalAddress(LOG2_E)); //0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL + movq(xmm4, Address(rsp, 8)); + pextrw(eax, xmm4, 3); + movl(edx, 8192); + movdl(xmm4, edx); + andl(eax, 32767); + subl(eax, 16); + jcc(Assembler::less, L_2TAG_PACKET_10_0_2); + movl(edx, eax); + andl(edx, 32752); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + addl(ecx, 16); + bsrl(ecx, ecx); + movl(tmp1, INT_MIN); + jmp(L_2TAG_PACKET_1_0_2); + + bind(L_2TAG_PACKET_34_0_2); + xorpd(xmm1, xmm1); + movl(eax, 32752); + pinsrw(xmm1, eax, 3); + xorpd(xmm0, xmm0); + mulsd(xmm0, xmm1); + movl(Address(rsp, 0), 28); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_35_0_2); + xorpd(xmm1, xmm1); + movl(edx, 30704); + pinsrw(xmm1, edx, 3); + movq(xmm2, ExternalAddress(LOG2_E)); //0x00000000UL, 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL + movq(xmm4, Address(rsp, 8)); + pextrw(eax, xmm4, 3); + movl(edx, 8192); + movdl(xmm4, edx); + andl(eax, 32767); + subl(eax, 16); + jcc(Assembler::less, L_2TAG_PACKET_8_0_2); + movl(edx, eax); + andl(edx, 32752); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + addl(ecx, 16); + bsrl(ecx, ecx); + movl(tmp1, 0); + jmp(L_2TAG_PACKET_1_0_2); + + bind(L_2TAG_PACKET_19_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_22_0_2); + xorpd(xmm0, xmm0); + jmp(B1_5); + + bind(L_2TAG_PACKET_11_0_2); + addl(eax, 384); + cmpl(eax, 0); + jcc(Assembler::less, L_2TAG_PACKET_38_0_2); + mulsd(xmm5, xmm1); + addsd(xmm0, xmm7); + shrl(tmp1, 31); + addpd(xmm3, xmm0); + pshufd(xmm0, xmm3, 238); + addsd(xmm3, xmm0); + lea(tmp4, ExternalAddress(log2)); //0xfefa39efUL, 0x3fe62e42UL, 0xfefa39efUL, 0xbfe62e42UL + movq(xmm4, Address(tmp4, tmp1, Address::times_8, 0)); + mulsd(xmm1, xmm3); + xorpd(xmm0, xmm0); + movl(eax, 16368); + shll(tmp1, 15); + orl(eax, tmp1); + pinsrw(xmm0, eax, 3); + addsd(xmm5, xmm1); + mulsd(xmm5, xmm4); + addsd(xmm0, xmm5); + jmp(B1_5); + + bind(L_2TAG_PACKET_38_0_2); + + bind(L_2TAG_PACKET_37_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_39_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + movl(Address(rsp, 0), 26); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_9_0_2); + movq(xmm1, Address(rsp, 16)); + movdqu(xmm2, xmm1); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + cmpl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_40_0_2); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_40_0_2); + movdl(eax, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_39_0_2); + shrl(edx, 21); + cmpl(edx, 1075); + jcc(Assembler::above, L_2TAG_PACKET_41_0_2); + jcc(Assembler::equal, L_2TAG_PACKET_42_0_2); + cmpl(edx, 1023); + jcc(Assembler::below, L_2TAG_PACKET_41_0_2); + movq(xmm1, Address(rsp, 16)); + movl(eax, 17208); + xorpd(xmm3, xmm3); + pinsrw(xmm3, eax, 3); + movdqu(xmm4, xmm3); + addsd(xmm3, xmm1); + subsd(xmm4, xmm3); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_41_0_2); + movdl(eax, xmm3); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_41_0_2); + + bind(L_2TAG_PACKET_43_0_2); + movq(xmm0, Address(rsp, 8)); + testl(ecx, INT_MIN); + jcc(Assembler::notEqual, L_2TAG_PACKET_44_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_42_0_2); + movq(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_43_0_2); + + bind(L_2TAG_PACKET_41_0_2); + testl(ecx, INT_MIN); + jcc(Assembler::equal, L_2TAG_PACKET_22_0_2); + xorpd(xmm0, xmm0); + + bind(L_2TAG_PACKET_44_0_2); + movl(eax, 16368); + xorpd(xmm1, xmm1); + pinsrw(xmm1, eax, 3); + divsd(xmm1, xmm0); + movdqu(xmm0, xmm1); + movl(Address(rsp, 0), 27); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_12_0_2); + movq(xmm2, Address(rsp, 8)); + movq(xmm6, Address(rsp, 16)); + pextrw(eax, xmm2, 3); + pextrw(edx, xmm6, 3); + movl(ecx, 32752); + andl(ecx, edx); + cmpl(ecx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_45_0_2); + andl(eax, 32752); + subl(eax, 16368); + xorl(edx, eax); + testl(edx, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_46_0_2); + + bind(L_2TAG_PACKET_47_0_2); + movl(eax, 32736); + pinsrw(xmm0, eax, 3); + shrl(tmp1, 16); + orl(eax, tmp1); + pinsrw(xmm1, eax, 3); + mulsd(xmm0, xmm1); + + bind(L_2TAG_PACKET_14_0_2); + movl(Address(rsp, 0), 24); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_46_0_2); + movl(eax, 16); + pinsrw(xmm0, eax, 3); + mulsd(xmm0, xmm0); + testl(tmp1, INT_MIN); + jcc(Assembler::equal, L_2TAG_PACKET_48_0_2); + mov64(tmp2, 0x8000000000000000); + movdq(xmm2, tmp2); + xorpd(xmm0, xmm2); + + bind(L_2TAG_PACKET_48_0_2); + movl(Address(rsp, 0), 25); + jmp(L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_13_0_2); + pextrw(ecx, xmm5, 3); + pextrw(edx, xmm4, 3); + movl(eax, -1); + andl(ecx, 32752); + subl(ecx, 16368); + andl(edx, 32752); + addl(edx, ecx); + movl(ecx, -31); + sarl(edx, 4); + subl(ecx, edx); + jcc(Assembler::lessEqual, L_2TAG_PACKET_49_0_2); + cmpl(ecx, 20); + jcc(Assembler::above, L_2TAG_PACKET_50_0_2); + shll(eax); + + bind(L_2TAG_PACKET_49_0_2); + movdl(xmm0, eax); + psllq(xmm0, 32); + pand(xmm0, xmm5); + subsd(xmm5, xmm0); + addsd(xmm5, xmm1); + mulsd(xmm0, xmm4); + mulsd(xmm5, xmm4); + addsd(xmm0, xmm5); + + bind(L_2TAG_PACKET_50_0_2); + jmp(L_2TAG_PACKET_48_0_2); + + bind(L_2TAG_PACKET_2_0_2); + movw(ecx, Address(rsp, 22)); + movl(edx, INT_MIN); + movdl(xmm1, rdx); + xorpd(xmm7, xmm7); + paddd(xmm0, xmm4); + movdl(edx, xmm0); + psllq(xmm0, 29); + paddq(xmm1, xmm3); + pand(xmm5, xmm1); + andl(ecx, 32752); + cmpl(ecx, 16560); + jcc(Assembler::less, L_2TAG_PACKET_3_0_2); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + addl(eax, 16351); + shrl(eax, 4); + subl(eax, 1022); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + lea(r11, ExternalAddress(L_tbl)); + movq(xmm4, ExternalAddress(coeff_h)); //0x00000000UL, 0xbfd61a00UL, 0x00000000UL, 0xbf5dabe1UL + mulsd(xmm3, xmm0); + movq(xmm6, ExternalAddress(coeff_h)); //0x00000000UL, 0xbfd61a00UL, 0x00000000UL, 0xbf5dabe1UL + subsd(xmm5, xmm2); + movq(xmm1, ExternalAddress(8 + coeff_h)); //0x00000000UL, 0xbf5dabe1UL + pshufd(xmm2, xmm3, 68); + unpcklpd(xmm5, xmm3); + addsd(xmm3, xmm5); + movq(xmm0, ExternalAddress(8 + coeff_h)); //0x00000000UL, 0xbf5dabe1UL + andl(edx, 16760832); + shrl(edx, 10); + addpd(xmm7, Address(tmp4, edx, Address::times_1, -3648)); + mulsd(xmm4, xmm5); + mulsd(xmm0, xmm5); + mulsd(xmm6, xmm2); + mulsd(xmm1, xmm2); + movdqu(xmm2, xmm5); + mulsd(xmm4, xmm5); + addsd(xmm5, xmm0); + movdqu(xmm0, xmm7); + addsd(xmm2, xmm3); + addsd(xmm7, xmm5); + mulsd(xmm6, xmm2); + subsd(xmm0, xmm7); + movdqu(xmm2, xmm7); + addsd(xmm7, xmm4); + addsd(xmm0, xmm5); + subsd(xmm2, xmm7); + addsd(xmm4, xmm2); + pshufd(xmm2, xmm5, 238); + movdqu(xmm5, xmm7); + addsd(xmm7, xmm2); + addsd(xmm4, xmm0); + movdqu(xmm0, ExternalAddress(coeff)); //0x6dc96112UL, 0xbf836578UL, 0xee241472UL, 0xbf9b0301UL + subsd(xmm5, xmm7); + addsd(xmm6, xmm4); + movdqu(xmm4, xmm7); + addsd(xmm5, xmm2); + addsd(xmm7, xmm1); + movdqu(xmm2, ExternalAddress(64 + coeff)); //0x486ececcUL, 0x3fc4635eUL, 0x161bb241UL, 0xbf5dabe1UL + subsd(xmm4, xmm7); + addsd(xmm6, xmm5); + addsd(xmm4, xmm1); + pshufd(xmm5, xmm7, 238); + movapd(xmm1, xmm7); + addsd(xmm7, xmm5); + subsd(xmm1, xmm7); + addsd(xmm1, xmm5); + movdqu(xmm5, ExternalAddress(80 + coeff)); //0x9f95985aUL, 0xbfb528dbUL, 0xf8b5787dUL, 0x3ef2531eUL + pshufd(xmm3, xmm3, 68); + addsd(xmm6, xmm4); + addsd(xmm6, xmm1); + movdqu(xmm1, ExternalAddress(32 + coeff)); //0x9f95985aUL, 0xbfb528dbUL, 0xb3841d2aUL, 0xbfd619b6UL + mulpd(xmm0, xmm3); + mulpd(xmm2, xmm3); + pshufd(xmm4, xmm3, 68); + mulpd(xmm3, xmm3); + addpd(xmm0, xmm1); + addpd(xmm5, xmm2); + mulsd(xmm4, xmm3); + movq(xmm2, ExternalAddress(HIGHMASK_LOG_X)); //0xf8000000UL, 0xffffffffUL, 0x00000000UL, 0xfffff800UL + mulpd(xmm3, xmm3); + movq(xmm1, Address(rsp, 16)); + movw(ecx, Address(rsp, 22)); + mulpd(xmm0, xmm4); + pextrw(eax, xmm7, 3); + mulpd(xmm5, xmm4); + mulpd(xmm0, xmm3); + movq(xmm4, ExternalAddress(8 + HIGHMASK_Y)); //0x00000000UL, 0xffffffffUL + pand(xmm2, xmm7); + addsd(xmm5, xmm6); + subsd(xmm7, xmm2); + addpd(xmm5, xmm0); + andl(eax, 32752); + subl(eax, 16368); + andl(ecx, 32752); + cmpl(ecx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_45_0_2); + addl(ecx, eax); + cmpl(ecx, 16576); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_51_0_2); + pshufd(xmm0, xmm5, 238); + pand(xmm4, xmm1); + movdqu(xmm3, xmm1); + addsd(xmm5, xmm0); + subsd(xmm1, xmm4); + xorpd(xmm6, xmm6); + movl(edx, 17080); + pinsrw(xmm6, edx, 3); + addsd(xmm7, xmm5); + mulsd(xmm4, xmm2); + mulsd(xmm1, xmm2); + movdqu(xmm5, xmm6); + mulsd(xmm3, xmm7); + addsd(xmm6, xmm4); + addsd(xmm1, xmm3); + movdqu(xmm7, ExternalAddress(e_coeff)); //0xe78a6731UL, 0x3f55d87fUL, 0xd704a0c0UL, 0x3fac6b08UL + movdl(edx, xmm6); + subsd(xmm6, xmm5); + lea(tmp4, ExternalAddress(T_exp)); + movdqu(xmm3, ExternalAddress(16 + e_coeff)); //0x6fba4e77UL, 0x3f83b2abUL, 0xff82c58fUL, 0x3fcebfbdUL + movq(xmm2, ExternalAddress(32 + e_coeff)); //0xfefa39efUL, 0x3fe62e42UL, 0x00000000UL, 0x00000000UL + subsd(xmm4, xmm6); + movl(ecx, edx); + andl(edx, 255); + addl(edx, edx); + movdqu(xmm5, Address(tmp4, edx, Address::times_8, 0)); + addsd(xmm4, xmm1); + pextrw(edx, xmm6, 3); + shrl(ecx, 8); + movl(eax, ecx); + shrl(ecx, 1); + subl(eax, ecx); + shll(ecx, 20); + movdl(xmm6, ecx); + pshufd(xmm0, xmm4, 68); + pshufd(xmm1, xmm4, 68); + mulpd(xmm0, xmm0); + mulpd(xmm7, xmm1); + pshufd(xmm6, xmm6, 17); + mulsd(xmm2, xmm4); + andl(edx, 32767); + cmpl(edx, 16529); + jcc(Assembler::above, L_2TAG_PACKET_12_0_2); + mulsd(xmm0, xmm0); + paddd(xmm5, xmm6); + addpd(xmm3, xmm7); + mulsd(xmm2, xmm5); + pshufd(xmm6, xmm5, 238); + mulpd(xmm0, xmm3); + addsd(xmm2, xmm6); + pshufd(xmm3, xmm0, 238); + addl(eax, 1023); + shll(eax, 20); + orl(eax, tmp1); + movdl(xmm4, eax); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + addsd(xmm0, xmm2); + psllq(xmm4, 32); + addsd(xmm0, xmm3); + movdqu(xmm1, xmm0); + addsd(xmm0, xmm5); + mulsd(xmm0, xmm4); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_13_0_2); + cmpl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_14_0_2); + + bind(L_2TAG_PACKET_52_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_45_0_2); + movq(xmm0, Address(rsp, 8)); + xorpd(xmm2, xmm2); + movl(eax, 49136); + pinsrw(xmm2, eax, 3); + addsd(xmm2, xmm0); + pextrw(eax, xmm2, 3); + cmpl(eax, 0); + jcc(Assembler::notEqual, L_2TAG_PACKET_53_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32760); + pinsrw(xmm0, eax, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_53_0_2); + movq(xmm1, Address(rsp, 16)); + movdl(edx, xmm1); + movdqu(xmm3, xmm1); + psrlq(xmm3, 20); + movdl(ecx, xmm3); + orl(ecx, edx); + jcc(Assembler::equal, L_2TAG_PACKET_54_0_2); + addsd(xmm1, xmm1); + movdqu(xmm0, xmm1); + jmp(B1_5); + + bind(L_2TAG_PACKET_51_0_2); + pextrw(eax, xmm1, 3); + pextrw(ecx, xmm2, 3); + xorl(eax, ecx); + testl(eax, 32768); + jcc(Assembler::equal, L_2TAG_PACKET_47_0_2); + jmp(L_2TAG_PACKET_46_0_2); + + bind(L_2TAG_PACKET_54_0_2); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + pextrw(edx, xmm1, 3); + xorpd(xmm0, xmm0); + subl(eax, 16368); + xorl(eax, edx); + testl(eax, 32768); + jcc(Assembler::equal, L_2TAG_PACKET_55_0_2); + jmp(B1_5); + + bind(L_2TAG_PACKET_55_0_2); + movl(edx, 32752); + pinsrw(xmm0, edx, 3); + jmp(B1_5); + + bind(L_2TAG_PACKET_17_0_2); + movq(Address(rsp, 24), xmm0); + + bind(B1_3); + movq(xmm0, Address(rsp, 24)); + + bind(L_2TAG_PACKET_56_0_2); + + bind(B1_5); + addq(rsp, 40); +} +#endif // _LP64 + +#ifndef _LP64 + +ALIGNED_(16) juint _static_const_table_pow[] = +{ + 0x00000000UL, 0xbfd61a00UL, 0x00000000UL, 0xbf5dabe1UL, 0xf8000000UL, + 0xffffffffUL, 0x00000000UL, 0xfffff800UL, 0x00000000UL, 0x3ff00000UL, + 0x00000000UL, 0x00000000UL, 0x20000000UL, 0x3feff00aUL, 0x96621f95UL, + 0x3e5b1856UL, 0xe0000000UL, 0x3fefe019UL, 0xe5916f9eUL, 0xbe325278UL, + 0x00000000UL, 0x3fefd02fUL, 0x859a1062UL, 0x3e595fb7UL, 0xc0000000UL, + 0x3fefc049UL, 0xb245f18fUL, 0xbe529c38UL, 0xe0000000UL, 0x3fefb069UL, + 0xad2880a7UL, 0xbe501230UL, 0x60000000UL, 0x3fefa08fUL, 0xc8e72420UL, + 0x3e597bd1UL, 0x80000000UL, 0x3fef90baUL, 0xc30c4500UL, 0xbe5d6c75UL, + 0xe0000000UL, 0x3fef80eaUL, 0x02c63f43UL, 0x3e2e1318UL, 0xc0000000UL, + 0x3fef7120UL, 0xb3d4ccccUL, 0xbe44c52aUL, 0x00000000UL, 0x3fef615cUL, + 0xdbd91397UL, 0xbe4e7d6cUL, 0xa0000000UL, 0x3fef519cUL, 0x65c5cd68UL, + 0xbe522dc8UL, 0xa0000000UL, 0x3fef41e2UL, 0x46d1306cUL, 0xbe5a840eUL, + 0xe0000000UL, 0x3fef322dUL, 0xd2980e94UL, 0x3e5071afUL, 0xa0000000UL, + 0x3fef227eUL, 0x773abadeUL, 0xbe5891e5UL, 0xa0000000UL, 0x3fef12d4UL, + 0xdc6bf46bUL, 0xbe5cccbeUL, 0xe0000000UL, 0x3fef032fUL, 0xbc7247faUL, + 0xbe2bab83UL, 0x80000000UL, 0x3feef390UL, 0xbcaa1e46UL, 0xbe53bb3bUL, + 0x60000000UL, 0x3feee3f6UL, 0x5f6c682dUL, 0xbe54c619UL, 0x80000000UL, + 0x3feed461UL, 0x5141e368UL, 0xbe4b6d86UL, 0xe0000000UL, 0x3feec4d1UL, + 0xec678f76UL, 0xbe369af6UL, 0x80000000UL, 0x3feeb547UL, 0x41301f55UL, + 0xbe2d4312UL, 0x60000000UL, 0x3feea5c2UL, 0x676da6bdUL, 0xbe4d8dd0UL, + 0x60000000UL, 0x3fee9642UL, 0x57a891c4UL, 0x3e51f991UL, 0xa0000000UL, + 0x3fee86c7UL, 0xe4eb491eUL, 0x3e579bf9UL, 0x20000000UL, 0x3fee7752UL, + 0xfddc4a2cUL, 0xbe3356e6UL, 0xc0000000UL, 0x3fee67e1UL, 0xd75b5bf1UL, + 0xbe449531UL, 0x80000000UL, 0x3fee5876UL, 0xbd423b8eUL, 0x3df54fe4UL, + 0x60000000UL, 0x3fee4910UL, 0x330e51b9UL, 0x3e54289cUL, 0x80000000UL, + 0x3fee39afUL, 0x8651a95fUL, 0xbe55aad6UL, 0xa0000000UL, 0x3fee2a53UL, + 0x5e98c708UL, 0xbe2fc4a9UL, 0xe0000000UL, 0x3fee1afcUL, 0x0989328dUL, + 0x3e23958cUL, 0x40000000UL, 0x3fee0babUL, 0xee642abdUL, 0xbe425dd8UL, + 0xa0000000UL, 0x3fedfc5eUL, 0xc394d236UL, 0x3e526362UL, 0x20000000UL, + 0x3feded17UL, 0xe104aa8eUL, 0x3e4ce247UL, 0xc0000000UL, 0x3fedddd4UL, + 0x265a9be4UL, 0xbe5bb77aUL, 0x40000000UL, 0x3fedce97UL, 0x0ecac52fUL, + 0x3e4a7cb1UL, 0xe0000000UL, 0x3fedbf5eUL, 0x124cb3b8UL, 0x3e257024UL, + 0x80000000UL, 0x3fedb02bUL, 0xe6d4febeUL, 0xbe2033eeUL, 0x20000000UL, + 0x3feda0fdUL, 0x39cca00eUL, 0xbe3ddabcUL, 0xc0000000UL, 0x3fed91d3UL, + 0xef8a552aUL, 0xbe543390UL, 0x40000000UL, 0x3fed82afUL, 0xb8e85204UL, + 0x3e513850UL, 0xe0000000UL, 0x3fed738fUL, 0x3d59fe08UL, 0xbe5db728UL, + 0x40000000UL, 0x3fed6475UL, 0x3aa7ead1UL, 0x3e58804bUL, 0xc0000000UL, + 0x3fed555fUL, 0xf8a35ba9UL, 0xbe5298b0UL, 0x00000000UL, 0x3fed464fUL, + 0x9a88dd15UL, 0x3e5a8cdbUL, 0x40000000UL, 0x3fed3743UL, 0xb0b0a190UL, + 0x3e598635UL, 0x80000000UL, 0x3fed283cUL, 0xe2113295UL, 0xbe5c1119UL, + 0x80000000UL, 0x3fed193aUL, 0xafbf1728UL, 0xbe492e9cUL, 0x60000000UL, + 0x3fed0a3dUL, 0xe4a4ccf3UL, 0x3e19b90eUL, 0x20000000UL, 0x3fecfb45UL, + 0xba3cbeb8UL, 0x3e406b50UL, 0xc0000000UL, 0x3fecec51UL, 0x110f7dddUL, + 0x3e0d6806UL, 0x40000000UL, 0x3fecdd63UL, 0x7dd7d508UL, 0xbe5a8943UL, + 0x80000000UL, 0x3fecce79UL, 0x9b60f271UL, 0xbe50676aUL, 0x80000000UL, + 0x3fecbf94UL, 0x0b9ad660UL, 0x3e59174fUL, 0x60000000UL, 0x3fecb0b4UL, + 0x00823d9cUL, 0x3e5bbf72UL, 0x20000000UL, 0x3feca1d9UL, 0x38a6ec89UL, + 0xbe4d38f9UL, 0x80000000UL, 0x3fec9302UL, 0x3a0b7d8eUL, 0x3e53dbfdUL, + 0xc0000000UL, 0x3fec8430UL, 0xc6826b34UL, 0xbe27c5c9UL, 0xc0000000UL, + 0x3fec7563UL, 0x0c706381UL, 0xbe593653UL, 0x60000000UL, 0x3fec669bUL, + 0x7df34ec7UL, 0x3e461ab5UL, 0xe0000000UL, 0x3fec57d7UL, 0x40e5e7e8UL, + 0xbe5c3daeUL, 0x00000000UL, 0x3fec4919UL, 0x5602770fUL, 0xbe55219dUL, + 0xc0000000UL, 0x3fec3a5eUL, 0xec7911ebUL, 0x3e5a5d25UL, 0x60000000UL, + 0x3fec2ba9UL, 0xb39ea225UL, 0xbe53c00bUL, 0x80000000UL, 0x3fec1cf8UL, + 0x967a212eUL, 0x3e5a8ddfUL, 0x60000000UL, 0x3fec0e4cUL, 0x580798bdUL, + 0x3e5f53abUL, 0x00000000UL, 0x3febffa5UL, 0xb8282df6UL, 0xbe46b874UL, + 0x20000000UL, 0x3febf102UL, 0xe33a6729UL, 0x3e54963fUL, 0x00000000UL, + 0x3febe264UL, 0x3b53e88aUL, 0xbe3adce1UL, 0x60000000UL, 0x3febd3caUL, + 0xc2585084UL, 0x3e5cde9fUL, 0x80000000UL, 0x3febc535UL, 0xa335c5eeUL, + 0xbe39fd9cUL, 0x20000000UL, 0x3febb6a5UL, 0x7325b04dUL, 0x3e42ba15UL, + 0x60000000UL, 0x3feba819UL, 0x1564540fUL, 0x3e3a9f35UL, 0x40000000UL, + 0x3feb9992UL, 0x83fff592UL, 0xbe5465ceUL, 0xa0000000UL, 0x3feb8b0fUL, + 0xb9da63d3UL, 0xbe4b1a0aUL, 0x80000000UL, 0x3feb7c91UL, 0x6d6f1ea4UL, + 0x3e557657UL, 0x00000000UL, 0x3feb6e18UL, 0x5e80a1bfUL, 0x3e4ddbb6UL, + 0x00000000UL, 0x3feb5fa3UL, 0x1c9eacb5UL, 0x3e592877UL, 0xa0000000UL, + 0x3feb5132UL, 0x6d40beb3UL, 0xbe51858cUL, 0xa0000000UL, 0x3feb42c6UL, + 0xd740c67bUL, 0x3e427ad2UL, 0x40000000UL, 0x3feb345fUL, 0xa3e0cceeUL, + 0xbe5c2fc4UL, 0x40000000UL, 0x3feb25fcUL, 0x8e752b50UL, 0xbe3da3c2UL, + 0xc0000000UL, 0x3feb179dUL, 0xa892e7deUL, 0x3e1fb481UL, 0xc0000000UL, + 0x3feb0943UL, 0x21ed71e9UL, 0xbe365206UL, 0x20000000UL, 0x3feafaeeUL, + 0x0e1380a3UL, 0x3e5c5b7bUL, 0x20000000UL, 0x3feaec9dUL, 0x3c3d640eUL, + 0xbe5dbbd0UL, 0x60000000UL, 0x3feade50UL, 0x8f97a715UL, 0x3e3a8ec5UL, + 0x20000000UL, 0x3fead008UL, 0x23ab2839UL, 0x3e2fe98aUL, 0x40000000UL, + 0x3feac1c4UL, 0xf4bbd50fUL, 0x3e54d8f6UL, 0xe0000000UL, 0x3feab384UL, + 0x14757c4dUL, 0xbe48774cUL, 0xc0000000UL, 0x3feaa549UL, 0x7c7b0eeaUL, + 0x3e5b51bbUL, 0x20000000UL, 0x3fea9713UL, 0xf56f7013UL, 0x3e386200UL, + 0xe0000000UL, 0x3fea88e0UL, 0xbe428ebeUL, 0xbe514af5UL, 0xe0000000UL, + 0x3fea7ab2UL, 0x8d0e4496UL, 0x3e4f9165UL, 0x60000000UL, 0x3fea6c89UL, + 0xdbacc5d5UL, 0xbe5c063bUL, 0x20000000UL, 0x3fea5e64UL, 0x3f19d970UL, + 0xbe5a0c8cUL, 0x20000000UL, 0x3fea5043UL, 0x09ea3e6bUL, 0x3e5065dcUL, + 0x80000000UL, 0x3fea4226UL, 0x78df246cUL, 0x3e5e05f6UL, 0x40000000UL, + 0x3fea340eUL, 0x4057d4a0UL, 0x3e431b2bUL, 0x40000000UL, 0x3fea25faUL, + 0x82867bb5UL, 0x3e4b76beUL, 0xa0000000UL, 0x3fea17eaUL, 0x9436f40aUL, + 0xbe5aad39UL, 0x20000000UL, 0x3fea09dfUL, 0x4b5253b3UL, 0x3e46380bUL, + 0x00000000UL, 0x3fe9fbd8UL, 0x8fc52466UL, 0xbe386f9bUL, 0x20000000UL, + 0x3fe9edd5UL, 0x22d3f344UL, 0xbe538347UL, 0x60000000UL, 0x3fe9dfd6UL, + 0x1ac33522UL, 0x3e5dbc53UL, 0x00000000UL, 0x3fe9d1dcUL, 0xeabdff1dUL, + 0x3e40fc0cUL, 0xe0000000UL, 0x3fe9c3e5UL, 0xafd30e73UL, 0xbe585e63UL, + 0xe0000000UL, 0x3fe9b5f3UL, 0xa52f226aUL, 0xbe43e8f9UL, 0x20000000UL, + 0x3fe9a806UL, 0xecb8698dUL, 0xbe515b36UL, 0x80000000UL, 0x3fe99a1cUL, + 0xf2b4e89dUL, 0x3e48b62bUL, 0x20000000UL, 0x3fe98c37UL, 0x7c9a88fbUL, + 0x3e44414cUL, 0x00000000UL, 0x3fe97e56UL, 0xda015741UL, 0xbe5d13baUL, + 0xe0000000UL, 0x3fe97078UL, 0x5fdace06UL, 0x3e51b947UL, 0x00000000UL, + 0x3fe962a0UL, 0x956ca094UL, 0x3e518785UL, 0x40000000UL, 0x3fe954cbUL, + 0x01164c1dUL, 0x3e5d5b57UL, 0xc0000000UL, 0x3fe946faUL, 0xe63b3767UL, + 0xbe4f84e7UL, 0x40000000UL, 0x3fe9392eUL, 0xe57cc2a9UL, 0x3e34eda3UL, + 0xe0000000UL, 0x3fe92b65UL, 0x8c75b544UL, 0x3e5766a0UL, 0xc0000000UL, + 0x3fe91da1UL, 0x37d1d087UL, 0xbe5e2ab1UL, 0x80000000UL, 0x3fe90fe1UL, + 0xa953dc20UL, 0x3e5fa1f3UL, 0x80000000UL, 0x3fe90225UL, 0xdbd3f369UL, + 0x3e47d6dbUL, 0xa0000000UL, 0x3fe8f46dUL, 0x1c9be989UL, 0xbe5e2b0aUL, + 0xa0000000UL, 0x3fe8e6b9UL, 0x3c93d76aUL, 0x3e5c8618UL, 0xe0000000UL, + 0x3fe8d909UL, 0x2182fc9aUL, 0xbe41aa9eUL, 0x20000000UL, 0x3fe8cb5eUL, + 0xe6b3539dUL, 0xbe530d19UL, 0x60000000UL, 0x3fe8bdb6UL, 0x49e58cc3UL, + 0xbe3bb374UL, 0xa0000000UL, 0x3fe8b012UL, 0xa7cfeb8fUL, 0x3e56c412UL, + 0x00000000UL, 0x3fe8a273UL, 0x8d52bc19UL, 0x3e1429b8UL, 0x60000000UL, + 0x3fe894d7UL, 0x4dc32c6cUL, 0xbe48604cUL, 0xc0000000UL, 0x3fe8873fUL, + 0x0c868e56UL, 0xbe564ee5UL, 0x00000000UL, 0x3fe879acUL, 0x56aee828UL, + 0x3e5e2fd8UL, 0x60000000UL, 0x3fe86c1cUL, 0x7ceab8ecUL, 0x3e493365UL, + 0xc0000000UL, 0x3fe85e90UL, 0x78d4dadcUL, 0xbe4f7f25UL, 0x00000000UL, + 0x3fe85109UL, 0x0ccd8280UL, 0x3e31e7a2UL, 0x40000000UL, 0x3fe84385UL, + 0x34ba4e15UL, 0x3e328077UL, 0x80000000UL, 0x3fe83605UL, 0xa670975aUL, + 0xbe53eee5UL, 0xa0000000UL, 0x3fe82889UL, 0xf61b77b2UL, 0xbe43a20aUL, + 0xa0000000UL, 0x3fe81b11UL, 0x13e6643bUL, 0x3e5e5fe5UL, 0xc0000000UL, + 0x3fe80d9dUL, 0x82cc94e8UL, 0xbe5ff1f9UL, 0xa0000000UL, 0x3fe8002dUL, + 0x8a0c9c5dUL, 0xbe42b0e7UL, 0x60000000UL, 0x3fe7f2c1UL, 0x22a16f01UL, + 0x3e5d9ea0UL, 0x20000000UL, 0x3fe7e559UL, 0xc38cd451UL, 0x3e506963UL, + 0xc0000000UL, 0x3fe7d7f4UL, 0x9902bc71UL, 0x3e4503d7UL, 0x40000000UL, + 0x3fe7ca94UL, 0xdef2a3c0UL, 0x3e3d98edUL, 0xa0000000UL, 0x3fe7bd37UL, + 0xed49abb0UL, 0x3e24c1ffUL, 0xe0000000UL, 0x3fe7afdeUL, 0xe3b0be70UL, + 0xbe40c467UL, 0x00000000UL, 0x3fe7a28aUL, 0xaf9f193cUL, 0xbe5dff6cUL, + 0xe0000000UL, 0x3fe79538UL, 0xb74cf6b6UL, 0xbe258ed0UL, 0xa0000000UL, + 0x3fe787ebUL, 0x1d9127c7UL, 0x3e345fb0UL, 0x40000000UL, 0x3fe77aa2UL, + 0x1028c21dUL, 0xbe4619bdUL, 0xa0000000UL, 0x3fe76d5cUL, 0x7cb0b5e4UL, + 0x3e40f1a2UL, 0xe0000000UL, 0x3fe7601aUL, 0x2b1bc4adUL, 0xbe32e8bbUL, + 0xe0000000UL, 0x3fe752dcUL, 0x6839f64eUL, 0x3e41f57bUL, 0xc0000000UL, + 0x3fe745a2UL, 0xc4121f7eUL, 0xbe52c40aUL, 0x60000000UL, 0x3fe7386cUL, + 0xd6852d72UL, 0xbe5c4e6bUL, 0xc0000000UL, 0x3fe72b39UL, 0x91d690f7UL, + 0xbe57f88fUL, 0xe0000000UL, 0x3fe71e0aUL, 0x627a2159UL, 0xbe4425d5UL, + 0xc0000000UL, 0x3fe710dfUL, 0x50a54033UL, 0x3e422b7eUL, 0x60000000UL, + 0x3fe703b8UL, 0x3b0b5f91UL, 0x3e5d3857UL, 0xe0000000UL, 0x3fe6f694UL, + 0x84d628a2UL, 0xbe51f090UL, 0x00000000UL, 0x3fe6e975UL, 0x306d8894UL, + 0xbe414d83UL, 0xe0000000UL, 0x3fe6dc58UL, 0x30bf24aaUL, 0xbe4650caUL, + 0x80000000UL, 0x3fe6cf40UL, 0xd4628d69UL, 0xbe5db007UL, 0xc0000000UL, + 0x3fe6c22bUL, 0xa2aae57bUL, 0xbe31d279UL, 0xc0000000UL, 0x3fe6b51aUL, + 0x860edf7eUL, 0xbe2d4c4aUL, 0x80000000UL, 0x3fe6a80dUL, 0xf3559341UL, + 0xbe5f7e98UL, 0xe0000000UL, 0x3fe69b03UL, 0xa885899eUL, 0xbe5c2011UL, + 0xe0000000UL, 0x3fe68dfdUL, 0x2bdc6d37UL, 0x3e224a82UL, 0xa0000000UL, + 0x3fe680fbUL, 0xc12ad1b9UL, 0xbe40cf56UL, 0x00000000UL, 0x3fe673fdUL, + 0x1bcdf659UL, 0xbdf52f2dUL, 0x00000000UL, 0x3fe66702UL, 0x5df10408UL, + 0x3e5663e0UL, 0xc0000000UL, 0x3fe65a0aUL, 0xa4070568UL, 0xbe40b12fUL, + 0x00000000UL, 0x3fe64d17UL, 0x71c54c47UL, 0x3e5f5e8bUL, 0x00000000UL, + 0x3fe64027UL, 0xbd4b7e83UL, 0x3e42ead6UL, 0xa0000000UL, 0x3fe6333aUL, + 0x61598bd2UL, 0xbe4c48d4UL, 0xc0000000UL, 0x3fe62651UL, 0x6f538d61UL, + 0x3e548401UL, 0xa0000000UL, 0x3fe6196cUL, 0x14344120UL, 0xbe529af6UL, + 0x00000000UL, 0x3fe60c8bUL, 0x5982c587UL, 0xbe3e1e4fUL, 0x00000000UL, + 0x3fe5ffadUL, 0xfe51d4eaUL, 0xbe4c897aUL, 0x80000000UL, 0x3fe5f2d2UL, + 0xfd46ebe1UL, 0x3e552e00UL, 0xa0000000UL, 0x3fe5e5fbUL, 0xa4695699UL, + 0x3e5ed471UL, 0x60000000UL, 0x3fe5d928UL, 0x80d118aeUL, 0x3e456b61UL, + 0xa0000000UL, 0x3fe5cc58UL, 0x304c330bUL, 0x3e54dc29UL, 0x80000000UL, + 0x3fe5bf8cUL, 0x0af2dedfUL, 0xbe3aa9bdUL, 0xe0000000UL, 0x3fe5b2c3UL, + 0x15fc9258UL, 0xbe479a37UL, 0xc0000000UL, 0x3fe5a5feUL, 0x9292c7eaUL, + 0x3e188650UL, 0x20000000UL, 0x3fe5993dUL, 0x33b4d380UL, 0x3e5d6d93UL, + 0x20000000UL, 0x3fe58c7fUL, 0x02fd16c7UL, 0x3e2fe961UL, 0xa0000000UL, + 0x3fe57fc4UL, 0x4a05edb6UL, 0xbe4d55b4UL, 0xa0000000UL, 0x3fe5730dUL, + 0x3d443abbUL, 0xbe5e6954UL, 0x00000000UL, 0x3fe5665aUL, 0x024acfeaUL, + 0x3e50e61bUL, 0x00000000UL, 0x3fe559aaUL, 0xcc9edd09UL, 0xbe325403UL, + 0x60000000UL, 0x3fe54cfdUL, 0x1fe26950UL, 0x3e5d500eUL, 0x60000000UL, + 0x3fe54054UL, 0x6c5ae164UL, 0xbe4a79b4UL, 0xc0000000UL, 0x3fe533aeUL, + 0x154b0287UL, 0xbe401571UL, 0xa0000000UL, 0x3fe5270cUL, 0x0673f401UL, + 0xbe56e56bUL, 0xe0000000UL, 0x3fe51a6dUL, 0x751b639cUL, 0x3e235269UL, + 0xa0000000UL, 0x3fe50dd2UL, 0x7c7b2bedUL, 0x3ddec887UL, 0xc0000000UL, + 0x3fe5013aUL, 0xafab4e17UL, 0x3e5e7575UL, 0x60000000UL, 0x3fe4f4a6UL, + 0x2e308668UL, 0x3e59aed6UL, 0x80000000UL, 0x3fe4e815UL, 0xf33e2a76UL, + 0xbe51f184UL, 0xe0000000UL, 0x3fe4db87UL, 0x839f3e3eUL, 0x3e57db01UL, + 0xc0000000UL, 0x3fe4cefdUL, 0xa9eda7bbUL, 0x3e535e0fUL, 0x00000000UL, + 0x3fe4c277UL, 0x2a8f66a5UL, 0x3e5ce451UL, 0xc0000000UL, 0x3fe4b5f3UL, + 0x05192456UL, 0xbe4e8518UL, 0xc0000000UL, 0x3fe4a973UL, 0x4aa7cd1dUL, + 0x3e46784aUL, 0x40000000UL, 0x3fe49cf7UL, 0x8e23025eUL, 0xbe5749f2UL, + 0x00000000UL, 0x3fe4907eUL, 0x18d30215UL, 0x3e360f39UL, 0x20000000UL, + 0x3fe48408UL, 0x63dcf2f3UL, 0x3e5e00feUL, 0xc0000000UL, 0x3fe47795UL, + 0x46182d09UL, 0xbe5173d9UL, 0xa0000000UL, 0x3fe46b26UL, 0x8f0e62aaUL, + 0xbe48f281UL, 0xe0000000UL, 0x3fe45ebaUL, 0x5775c40cUL, 0xbe56aad4UL, + 0x60000000UL, 0x3fe45252UL, 0x0fe25f69UL, 0x3e48bd71UL, 0x40000000UL, + 0x3fe445edUL, 0xe9989ec5UL, 0x3e590d97UL, 0x80000000UL, 0x3fe4398bUL, + 0xb3d9ffe3UL, 0x3e479dbcUL, 0x20000000UL, 0x3fe42d2dUL, 0x388e4d2eUL, + 0xbe5eed80UL, 0xe0000000UL, 0x3fe420d1UL, 0x6f797c18UL, 0x3e554b4cUL, + 0x20000000UL, 0x3fe4147aUL, 0x31048bb4UL, 0xbe5b1112UL, 0x80000000UL, + 0x3fe40825UL, 0x2efba4f9UL, 0x3e48ebc7UL, 0x40000000UL, 0x3fe3fbd4UL, + 0x50201119UL, 0x3e40b701UL, 0x40000000UL, 0x3fe3ef86UL, 0x0a4db32cUL, + 0x3e551de8UL, 0xa0000000UL, 0x3fe3e33bUL, 0x0c9c148bUL, 0xbe50c1f6UL, + 0x20000000UL, 0x3fe3d6f4UL, 0xc9129447UL, 0x3e533fa0UL, 0x00000000UL, + 0x3fe3cab0UL, 0xaae5b5a0UL, 0xbe22b68eUL, 0x20000000UL, 0x3fe3be6fUL, + 0x02305e8aUL, 0xbe54fc08UL, 0x60000000UL, 0x3fe3b231UL, 0x7f908258UL, + 0x3e57dc05UL, 0x00000000UL, 0x3fe3a5f7UL, 0x1a09af78UL, 0x3e08038bUL, + 0xe0000000UL, 0x3fe399bfUL, 0x490643c1UL, 0xbe5dbe42UL, 0xe0000000UL, + 0x3fe38d8bUL, 0x5e8ad724UL, 0xbe3c2b72UL, 0x20000000UL, 0x3fe3815bUL, + 0xc67196b6UL, 0x3e1713cfUL, 0xa0000000UL, 0x3fe3752dUL, 0x6182e429UL, + 0xbe3ec14cUL, 0x40000000UL, 0x3fe36903UL, 0xab6eb1aeUL, 0x3e5a2cc5UL, + 0x40000000UL, 0x3fe35cdcUL, 0xfe5dc064UL, 0xbe5c5878UL, 0x40000000UL, + 0x3fe350b8UL, 0x0ba6b9e4UL, 0x3e51619bUL, 0x80000000UL, 0x3fe34497UL, + 0x857761aaUL, 0x3e5fff53UL, 0x00000000UL, 0x3fe3387aUL, 0xf872d68cUL, + 0x3e484f4dUL, 0xa0000000UL, 0x3fe32c5fUL, 0x087e97c2UL, 0x3e52842eUL, + 0x80000000UL, 0x3fe32048UL, 0x73d6d0c0UL, 0xbe503edfUL, 0x80000000UL, + 0x3fe31434UL, 0x0c1456a1UL, 0xbe5f72adUL, 0xa0000000UL, 0x3fe30823UL, + 0x83a1a4d5UL, 0xbe5e65ccUL, 0xe0000000UL, 0x3fe2fc15UL, 0x855a7390UL, + 0xbe506438UL, 0x40000000UL, 0x3fe2f00bUL, 0xa2898287UL, 0x3e3d22a2UL, + 0xe0000000UL, 0x3fe2e403UL, 0x8b56f66fUL, 0xbe5aa5fdUL, 0x80000000UL, + 0x3fe2d7ffUL, 0x52db119aUL, 0x3e3a2e3dUL, 0x60000000UL, 0x3fe2cbfeUL, + 0xe2ddd4c0UL, 0xbe586469UL, 0x40000000UL, 0x3fe2c000UL, 0x6b01bf10UL, + 0x3e352b9dUL, 0x40000000UL, 0x3fe2b405UL, 0xb07a1cdfUL, 0x3e5c5cdaUL, + 0x80000000UL, 0x3fe2a80dUL, 0xc7b5f868UL, 0xbe5668b3UL, 0xc0000000UL, + 0x3fe29c18UL, 0x185edf62UL, 0xbe563d66UL, 0x00000000UL, 0x3fe29027UL, + 0xf729e1ccUL, 0x3e59a9a0UL, 0x80000000UL, 0x3fe28438UL, 0x6433c727UL, + 0xbe43cc89UL, 0x00000000UL, 0x3fe2784dUL, 0x41782631UL, 0xbe30750cUL, + 0xa0000000UL, 0x3fe26c64UL, 0x914911b7UL, 0xbe58290eUL, 0x40000000UL, + 0x3fe2607fUL, 0x3dcc73e1UL, 0xbe4269cdUL, 0x00000000UL, 0x3fe2549dUL, + 0x2751bf70UL, 0xbe5a6998UL, 0xc0000000UL, 0x3fe248bdUL, 0x4248b9fbUL, + 0xbe4ddb00UL, 0x80000000UL, 0x3fe23ce1UL, 0xf35cf82fUL, 0x3e561b71UL, + 0x60000000UL, 0x3fe23108UL, 0x8e481a2dUL, 0x3e518fb9UL, 0x60000000UL, + 0x3fe22532UL, 0x5ab96edcUL, 0xbe5fafc5UL, 0x40000000UL, 0x3fe2195fUL, + 0x80943911UL, 0xbe07f819UL, 0x40000000UL, 0x3fe20d8fUL, 0x386f2d6cUL, + 0xbe54ba8bUL, 0x40000000UL, 0x3fe201c2UL, 0xf29664acUL, 0xbe5eb815UL, + 0x20000000UL, 0x3fe1f5f8UL, 0x64f03390UL, 0x3e5e320cUL, 0x20000000UL, + 0x3fe1ea31UL, 0x747ff696UL, 0x3e5ef0a5UL, 0x40000000UL, 0x3fe1de6dUL, + 0x3e9ceb51UL, 0xbe5f8d27UL, 0x20000000UL, 0x3fe1d2acUL, 0x4ae0b55eUL, + 0x3e5faa21UL, 0x20000000UL, 0x3fe1c6eeUL, 0x28569a5eUL, 0x3e598a4fUL, + 0x20000000UL, 0x3fe1bb33UL, 0x54b33e07UL, 0x3e46130aUL, 0x20000000UL, + 0x3fe1af7bUL, 0x024f1078UL, 0xbe4dbf93UL, 0x00000000UL, 0x3fe1a3c6UL, + 0xb0783bfaUL, 0x3e419248UL, 0xe0000000UL, 0x3fe19813UL, 0x2f02b836UL, + 0x3e4e02b7UL, 0xc0000000UL, 0x3fe18c64UL, 0x28dec9d4UL, 0x3e09064fUL, + 0x80000000UL, 0x3fe180b8UL, 0x45cbf406UL, 0x3e5b1f46UL, 0x40000000UL, + 0x3fe1750fUL, 0x03d9964cUL, 0x3e5b0a79UL, 0x00000000UL, 0x3fe16969UL, + 0x8b5b882bUL, 0xbe238086UL, 0xa0000000UL, 0x3fe15dc5UL, 0x73bad6f8UL, + 0xbdf1fca4UL, 0x20000000UL, 0x3fe15225UL, 0x5385769cUL, 0x3e5e8d76UL, + 0xa0000000UL, 0x3fe14687UL, 0x1676dc6bUL, 0x3e571d08UL, 0x20000000UL, + 0x3fe13aedUL, 0xa8c41c7fUL, 0xbe598a25UL, 0x60000000UL, 0x3fe12f55UL, + 0xc4e1aaf0UL, 0x3e435277UL, 0xa0000000UL, 0x3fe123c0UL, 0x403638e1UL, + 0xbe21aa7cUL, 0xc0000000UL, 0x3fe1182eUL, 0x557a092bUL, 0xbdd0116bUL, + 0xc0000000UL, 0x3fe10c9fUL, 0x7d779f66UL, 0x3e4a61baUL, 0xc0000000UL, + 0x3fe10113UL, 0x2b09c645UL, 0xbe5d586eUL, 0x20000000UL, 0x3fe0ea04UL, + 0xea2cad46UL, 0x3e5aa97cUL, 0x20000000UL, 0x3fe0d300UL, 0x23190e54UL, + 0x3e50f1a7UL, 0xa0000000UL, 0x3fe0bc07UL, 0x1379a5a6UL, 0xbe51619dUL, + 0x60000000UL, 0x3fe0a51aUL, 0x926a3d4aUL, 0x3e5cf019UL, 0xa0000000UL, + 0x3fe08e38UL, 0xa8c24358UL, 0x3e35241eUL, 0x20000000UL, 0x3fe07762UL, + 0x24317e7aUL, 0x3e512cfaUL, 0x00000000UL, 0x3fe06097UL, 0xfd9cf274UL, + 0xbe55bef3UL, 0x00000000UL, 0x3fe049d7UL, 0x3689b49dUL, 0xbe36d26dUL, + 0x40000000UL, 0x3fe03322UL, 0xf72ef6c4UL, 0xbe54cd08UL, 0xa0000000UL, + 0x3fe01c78UL, 0x23702d2dUL, 0xbe5900bfUL, 0x00000000UL, 0x3fe005daUL, + 0x3f59c14cUL, 0x3e57d80bUL, 0x40000000UL, 0x3fdfde8dUL, 0xad67766dUL, + 0xbe57fad4UL, 0x40000000UL, 0x3fdfb17cUL, 0x644f4ae7UL, 0x3e1ee43bUL, + 0x40000000UL, 0x3fdf8481UL, 0x903234d2UL, 0x3e501a86UL, 0x40000000UL, + 0x3fdf579cUL, 0xafe9e509UL, 0xbe267c3eUL, 0x00000000UL, 0x3fdf2acdUL, + 0xb7dfda0bUL, 0xbe48149bUL, 0x40000000UL, 0x3fdefe13UL, 0x3b94305eUL, + 0x3e5f4ea7UL, 0x80000000UL, 0x3fded16fUL, 0x5d95da61UL, 0xbe55c198UL, + 0x00000000UL, 0x3fdea4e1UL, 0x406960c9UL, 0xbdd99a19UL, 0x00000000UL, + 0x3fde7868UL, 0xd22f3539UL, 0x3e470c78UL, 0x80000000UL, 0x3fde4c04UL, + 0x83eec535UL, 0xbe3e1232UL, 0x40000000UL, 0x3fde1fb6UL, 0x3dfbffcbUL, + 0xbe4b7d71UL, 0x40000000UL, 0x3fddf37dUL, 0x7e1be4e0UL, 0xbe5b8f8fUL, + 0x40000000UL, 0x3fddc759UL, 0x46dae887UL, 0xbe350458UL, 0x80000000UL, + 0x3fdd9b4aUL, 0xed6ecc49UL, 0xbe5f0045UL, 0x80000000UL, 0x3fdd6f50UL, + 0x2e9e883cUL, 0x3e2915daUL, 0x80000000UL, 0x3fdd436bUL, 0xf0bccb32UL, + 0x3e4a68c9UL, 0x80000000UL, 0x3fdd179bUL, 0x9bbfc779UL, 0xbe54a26aUL, + 0x00000000UL, 0x3fdcebe0UL, 0x7cea33abUL, 0x3e43c6b7UL, 0x40000000UL, + 0x3fdcc039UL, 0xe740fd06UL, 0x3e5526c2UL, 0x40000000UL, 0x3fdc94a7UL, + 0x9eadeb1aUL, 0xbe396d8dUL, 0xc0000000UL, 0x3fdc6929UL, 0xf0a8f95aUL, + 0xbe5c0ab2UL, 0x80000000UL, 0x3fdc3dc0UL, 0x6ee2693bUL, 0x3e0992e6UL, + 0xc0000000UL, 0x3fdc126bUL, 0x5ac6b581UL, 0xbe2834b6UL, 0x40000000UL, + 0x3fdbe72bUL, 0x8cc226ffUL, 0x3e3596a6UL, 0x00000000UL, 0x3fdbbbffUL, + 0xf92a74bbUL, 0x3e3c5813UL, 0x00000000UL, 0x3fdb90e7UL, 0x479664c0UL, + 0xbe50d644UL, 0x00000000UL, 0x3fdb65e3UL, 0x5004975bUL, 0xbe55258fUL, + 0x00000000UL, 0x3fdb3af3UL, 0xe4b23194UL, 0xbe588407UL, 0xc0000000UL, + 0x3fdb1016UL, 0xe65d4d0aUL, 0x3e527c26UL, 0x80000000UL, 0x3fdae54eUL, + 0x814fddd6UL, 0x3e5962a2UL, 0x40000000UL, 0x3fdaba9aUL, 0xe19d0913UL, + 0xbe562f4eUL, 0x80000000UL, 0x3fda8ff9UL, 0x43cfd006UL, 0xbe4cfdebUL, + 0x40000000UL, 0x3fda656cUL, 0x686f0a4eUL, 0x3e5e47a8UL, 0xc0000000UL, + 0x3fda3af2UL, 0x7200d410UL, 0x3e5e1199UL, 0xc0000000UL, 0x3fda108cUL, + 0xabd2266eUL, 0x3e5ee4d1UL, 0x40000000UL, 0x3fd9e63aUL, 0x396f8f2cUL, + 0x3e4dbffbUL, 0x00000000UL, 0x3fd9bbfbUL, 0xe32b25ddUL, 0x3e5c3a54UL, + 0x40000000UL, 0x3fd991cfUL, 0x431e4035UL, 0xbe457925UL, 0x80000000UL, + 0x3fd967b6UL, 0x7bed3dd3UL, 0x3e40c61dUL, 0x00000000UL, 0x3fd93db1UL, + 0xd7449365UL, 0x3e306419UL, 0x80000000UL, 0x3fd913beUL, 0x1746e791UL, + 0x3e56fcfcUL, 0x40000000UL, 0x3fd8e9dfUL, 0xf3a9028bUL, 0xbe5041b9UL, + 0xc0000000UL, 0x3fd8c012UL, 0x56840c50UL, 0xbe26e20aUL, 0x40000000UL, + 0x3fd89659UL, 0x19763102UL, 0xbe51f466UL, 0x80000000UL, 0x3fd86cb2UL, + 0x7032de7cUL, 0xbe4d298aUL, 0x80000000UL, 0x3fd8431eUL, 0xdeb39fabUL, + 0xbe4361ebUL, 0x40000000UL, 0x3fd8199dUL, 0x5d01cbe0UL, 0xbe5425b3UL, + 0x80000000UL, 0x3fd7f02eUL, 0x3ce99aa9UL, 0x3e146fa8UL, 0x80000000UL, + 0x3fd7c6d2UL, 0xd1a262b9UL, 0xbe5a1a69UL, 0xc0000000UL, 0x3fd79d88UL, + 0x8606c236UL, 0x3e423a08UL, 0x80000000UL, 0x3fd77451UL, 0x8fd1e1b7UL, + 0x3e5a6a63UL, 0xc0000000UL, 0x3fd74b2cUL, 0xe491456aUL, 0x3e42c1caUL, + 0x40000000UL, 0x3fd7221aUL, 0x4499a6d7UL, 0x3e36a69aUL, 0x00000000UL, + 0x3fd6f91aUL, 0x5237df94UL, 0xbe0f8f02UL, 0x00000000UL, 0x3fd6d02cUL, + 0xb6482c6eUL, 0xbe5abcf7UL, 0x00000000UL, 0x3fd6a750UL, 0x1919fd61UL, + 0xbe57ade2UL, 0x00000000UL, 0x3fd67e86UL, 0xaa7a994dUL, 0xbe3f3fbdUL, + 0x00000000UL, 0x3fd655ceUL, 0x67db014cUL, 0x3e33c550UL, 0x00000000UL, + 0x3fd62d28UL, 0xa82856b7UL, 0xbe1409d1UL, 0xc0000000UL, 0x3fd60493UL, + 0x1e6a300dUL, 0x3e55d899UL, 0x80000000UL, 0x3fd5dc11UL, 0x1222bd5cUL, + 0xbe35bfc0UL, 0xc0000000UL, 0x3fd5b3a0UL, 0x6e8dc2d3UL, 0x3e5d4d79UL, + 0x00000000UL, 0x3fd58b42UL, 0xe0e4ace6UL, 0xbe517303UL, 0x80000000UL, + 0x3fd562f4UL, 0xb306e0a8UL, 0x3e5edf0fUL, 0xc0000000UL, 0x3fd53ab8UL, + 0x6574bc54UL, 0x3e5ee859UL, 0x80000000UL, 0x3fd5128eUL, 0xea902207UL, + 0x3e5f6188UL, 0xc0000000UL, 0x3fd4ea75UL, 0x9f911d79UL, 0x3e511735UL, + 0x80000000UL, 0x3fd4c26eUL, 0xf9c77397UL, 0xbe5b1643UL, 0x40000000UL, + 0x3fd49a78UL, 0x15fc9258UL, 0x3e479a37UL, 0x80000000UL, 0x3fd47293UL, + 0xd5a04dd9UL, 0xbe426e56UL, 0xc0000000UL, 0x3fd44abfUL, 0xe04042f5UL, + 0x3e56f7c6UL, 0x40000000UL, 0x3fd422fdUL, 0x1d8bf2c8UL, 0x3e5d8810UL, + 0x00000000UL, 0x3fd3fb4cUL, 0x88a8ddeeUL, 0xbe311454UL, 0xc0000000UL, + 0x3fd3d3abUL, 0x3e3b5e47UL, 0xbe5d1b72UL, 0x40000000UL, 0x3fd3ac1cUL, + 0xc2ab5d59UL, 0x3e31b02bUL, 0xc0000000UL, 0x3fd3849dUL, 0xd4e34b9eUL, + 0x3e51cb2fUL, 0x40000000UL, 0x3fd35d30UL, 0x177204fbUL, 0xbe2b8cd7UL, + 0x80000000UL, 0x3fd335d3UL, 0xfcd38c82UL, 0xbe4356e1UL, 0x80000000UL, + 0x3fd30e87UL, 0x64f54accUL, 0xbe4e6224UL, 0x00000000UL, 0x3fd2e74cUL, + 0xaa7975d9UL, 0x3e5dc0feUL, 0x80000000UL, 0x3fd2c021UL, 0x516dab3fUL, + 0xbe50ffa3UL, 0x40000000UL, 0x3fd29907UL, 0x2bfb7313UL, 0x3e5674a2UL, + 0xc0000000UL, 0x3fd271fdUL, 0x0549fc99UL, 0x3e385d29UL, 0xc0000000UL, + 0x3fd24b04UL, 0x55b63073UL, 0xbe500c6dUL, 0x00000000UL, 0x3fd2241cUL, + 0x3f91953aUL, 0x3e389977UL, 0xc0000000UL, 0x3fd1fd43UL, 0xa1543f71UL, + 0xbe3487abUL, 0xc0000000UL, 0x3fd1d67bUL, 0x4ec8867cUL, 0x3df6a2dcUL, + 0x00000000UL, 0x3fd1afc4UL, 0x4328e3bbUL, 0x3e41d9c0UL, 0x80000000UL, + 0x3fd1891cUL, 0x2e1cda84UL, 0x3e3bdd87UL, 0x40000000UL, 0x3fd16285UL, + 0x4b5331aeUL, 0xbe53128eUL, 0x00000000UL, 0x3fd13bfeUL, 0xb9aec164UL, + 0xbe52ac98UL, 0xc0000000UL, 0x3fd11586UL, 0xd91e1316UL, 0xbe350630UL, + 0x80000000UL, 0x3fd0ef1fUL, 0x7cacc12cUL, 0x3e3f5219UL, 0x40000000UL, + 0x3fd0c8c8UL, 0xbce277b7UL, 0x3e3d30c0UL, 0x00000000UL, 0x3fd0a281UL, + 0x2a63447dUL, 0xbe541377UL, 0x80000000UL, 0x3fd07c49UL, 0xfac483b5UL, + 0xbe5772ecUL, 0xc0000000UL, 0x3fd05621UL, 0x36b8a570UL, 0xbe4fd4bdUL, + 0xc0000000UL, 0x3fd03009UL, 0xbae505f7UL, 0xbe450388UL, 0x80000000UL, + 0x3fd00a01UL, 0x3e35aeadUL, 0xbe5430fcUL, 0x80000000UL, 0x3fcfc811UL, + 0x707475acUL, 0x3e38806eUL, 0x80000000UL, 0x3fcf7c3fUL, 0xc91817fcUL, + 0xbe40cceaUL, 0x80000000UL, 0x3fcf308cUL, 0xae05d5e9UL, 0xbe4919b8UL, + 0x80000000UL, 0x3fcee4f8UL, 0xae6cc9e6UL, 0xbe530b94UL, 0x00000000UL, + 0x3fce9983UL, 0x1efe3e8eUL, 0x3e57747eUL, 0x00000000UL, 0x3fce4e2dUL, + 0xda78d9bfUL, 0xbe59a608UL, 0x00000000UL, 0x3fce02f5UL, 0x8abe2c2eUL, + 0x3e4a35adUL, 0x00000000UL, 0x3fcdb7dcUL, 0x1495450dUL, 0xbe0872ccUL, + 0x80000000UL, 0x3fcd6ce1UL, 0x86ee0ba0UL, 0xbe4f59a0UL, 0x00000000UL, + 0x3fcd2205UL, 0xe81ca888UL, 0x3e5402c3UL, 0x00000000UL, 0x3fccd747UL, + 0x3b4424b9UL, 0x3e5dfdc3UL, 0x80000000UL, 0x3fcc8ca7UL, 0xd305b56cUL, + 0x3e202da6UL, 0x00000000UL, 0x3fcc4226UL, 0x399a6910UL, 0xbe482a1cUL, + 0x80000000UL, 0x3fcbf7c2UL, 0x747f7938UL, 0xbe587372UL, 0x80000000UL, + 0x3fcbad7cUL, 0x6fc246a0UL, 0x3e50d83dUL, 0x00000000UL, 0x3fcb6355UL, + 0xee9e9be5UL, 0xbe5c35bdUL, 0x80000000UL, 0x3fcb194aUL, 0x8416c0bcUL, + 0x3e546d4fUL, 0x00000000UL, 0x3fcacf5eUL, 0x49f7f08fUL, 0x3e56da76UL, + 0x00000000UL, 0x3fca858fUL, 0x5dc30de2UL, 0x3e5f390cUL, 0x00000000UL, + 0x3fca3bdeUL, 0x950583b6UL, 0xbe5e4169UL, 0x80000000UL, 0x3fc9f249UL, + 0x33631553UL, 0x3e52aeb1UL, 0x00000000UL, 0x3fc9a8d3UL, 0xde8795a6UL, + 0xbe59a504UL, 0x00000000UL, 0x3fc95f79UL, 0x076bf41eUL, 0x3e5122feUL, + 0x80000000UL, 0x3fc9163cUL, 0x2914c8e7UL, 0x3e3dd064UL, 0x00000000UL, + 0x3fc8cd1dUL, 0x3a30eca3UL, 0xbe21b4aaUL, 0x80000000UL, 0x3fc8841aUL, + 0xb2a96650UL, 0xbe575444UL, 0x80000000UL, 0x3fc83b34UL, 0x2376c0cbUL, + 0xbe2a74c7UL, 0x80000000UL, 0x3fc7f26bUL, 0xd8a0b653UL, 0xbe5181b6UL, + 0x00000000UL, 0x3fc7a9bfUL, 0x32257882UL, 0xbe4a78b4UL, 0x00000000UL, + 0x3fc7612fUL, 0x1eee8bd9UL, 0xbe1bfe9dUL, 0x80000000UL, 0x3fc718bbUL, + 0x0c603cc4UL, 0x3e36fdc9UL, 0x80000000UL, 0x3fc6d064UL, 0x3728b8cfUL, + 0xbe1e542eUL, 0x80000000UL, 0x3fc68829UL, 0xc79a4067UL, 0x3e5c380fUL, + 0x00000000UL, 0x3fc6400bUL, 0xf69eac69UL, 0x3e550a84UL, 0x80000000UL, + 0x3fc5f808UL, 0xb7a780a4UL, 0x3e5d9224UL, 0x80000000UL, 0x3fc5b022UL, + 0xad9dfb1eUL, 0xbe55242fUL, 0x00000000UL, 0x3fc56858UL, 0x659b18beUL, + 0xbe4bfda3UL, 0x80000000UL, 0x3fc520a9UL, 0x66ee3631UL, 0xbe57d769UL, + 0x80000000UL, 0x3fc4d916UL, 0x1ec62819UL, 0x3e2427f7UL, 0x80000000UL, + 0x3fc4919fUL, 0xdec25369UL, 0xbe435431UL, 0x00000000UL, 0x3fc44a44UL, + 0xa8acfc4bUL, 0xbe3c62e8UL, 0x00000000UL, 0x3fc40304UL, 0xcf1d3eabUL, + 0xbdfba29fUL, 0x80000000UL, 0x3fc3bbdfUL, 0x79aba3eaUL, 0xbdf1b7c8UL, + 0x80000000UL, 0x3fc374d6UL, 0xb8d186daUL, 0xbe5130cfUL, 0x80000000UL, + 0x3fc32de8UL, 0x9d74f152UL, 0x3e2285b6UL, 0x00000000UL, 0x3fc2e716UL, + 0x50ae7ca9UL, 0xbe503920UL, 0x80000000UL, 0x3fc2a05eUL, 0x6caed92eUL, + 0xbe533924UL, 0x00000000UL, 0x3fc259c2UL, 0x9cb5034eUL, 0xbe510e31UL, + 0x80000000UL, 0x3fc21340UL, 0x12c4d378UL, 0xbe540b43UL, 0x80000000UL, + 0x3fc1ccd9UL, 0xcc418706UL, 0x3e59887aUL, 0x00000000UL, 0x3fc1868eUL, + 0x921f4106UL, 0xbe528e67UL, 0x80000000UL, 0x3fc1405cUL, 0x3969441eUL, + 0x3e5d8051UL, 0x00000000UL, 0x3fc0fa46UL, 0xd941ef5bUL, 0x3e5f9079UL, + 0x80000000UL, 0x3fc0b44aUL, 0x5a3e81b2UL, 0xbe567691UL, 0x00000000UL, + 0x3fc06e69UL, 0x9d66afe7UL, 0xbe4d43fbUL, 0x00000000UL, 0x3fc028a2UL, + 0x0a92a162UL, 0xbe52f394UL, 0x00000000UL, 0x3fbfc5eaUL, 0x209897e5UL, + 0x3e529e37UL, 0x00000000UL, 0x3fbf3ac5UL, 0x8458bd7bUL, 0x3e582831UL, + 0x00000000UL, 0x3fbeafd5UL, 0xb8d8b4b8UL, 0xbe486b4aUL, 0x00000000UL, + 0x3fbe2518UL, 0xe0a3b7b6UL, 0x3e5bafd2UL, 0x00000000UL, 0x3fbd9a90UL, + 0x2bf2710eUL, 0x3e383b2bUL, 0x00000000UL, 0x3fbd103cUL, 0x73eb6ab7UL, + 0xbe56d78dUL, 0x00000000UL, 0x3fbc861bUL, 0x32ceaff5UL, 0xbe32dc5aUL, + 0x00000000UL, 0x3fbbfc2eUL, 0xbee04cb7UL, 0xbe4a71a4UL, 0x00000000UL, + 0x3fbb7274UL, 0x35ae9577UL, 0x3e38142fUL, 0x00000000UL, 0x3fbae8eeUL, + 0xcbaddab4UL, 0xbe5490f0UL, 0x00000000UL, 0x3fba5f9aUL, 0x95ce1114UL, + 0x3e597c71UL, 0x00000000UL, 0x3fb9d67aUL, 0x6d7c0f78UL, 0x3e3abc2dUL, + 0x00000000UL, 0x3fb94d8dUL, 0x2841a782UL, 0xbe566cbcUL, 0x00000000UL, + 0x3fb8c4d2UL, 0x6ed429c6UL, 0xbe3cfff9UL, 0x00000000UL, 0x3fb83c4aUL, + 0xe4a49fbbUL, 0xbe552964UL, 0x00000000UL, 0x3fb7b3f4UL, 0x2193d81eUL, + 0xbe42fa72UL, 0x00000000UL, 0x3fb72bd0UL, 0xdd70c122UL, 0x3e527a8cUL, + 0x00000000UL, 0x3fb6a3dfUL, 0x03108a54UL, 0xbe450393UL, 0x00000000UL, + 0x3fb61c1fUL, 0x30ff7954UL, 0x3e565840UL, 0x00000000UL, 0x3fb59492UL, + 0xdedd460cUL, 0xbe5422b5UL, 0x00000000UL, 0x3fb50d36UL, 0x950f9f45UL, + 0xbe5313f6UL, 0x00000000UL, 0x3fb4860bUL, 0x582cdcb1UL, 0x3e506d39UL, + 0x00000000UL, 0x3fb3ff12UL, 0x7216d3a6UL, 0x3e4aa719UL, 0x00000000UL, + 0x3fb3784aUL, 0x57a423fdUL, 0x3e5a9b9fUL, 0x00000000UL, 0x3fb2f1b4UL, + 0x7a138b41UL, 0xbe50b418UL, 0x00000000UL, 0x3fb26b4eUL, 0x2fbfd7eaUL, + 0x3e23a53eUL, 0x00000000UL, 0x3fb1e519UL, 0x18913ccbUL, 0x3e465fc1UL, + 0x00000000UL, 0x3fb15f15UL, 0x7ea24e21UL, 0x3e042843UL, 0x00000000UL, + 0x3fb0d941UL, 0x7c6d9c77UL, 0x3e59f61eUL, 0x00000000UL, 0x3fb0539eUL, + 0x114efd44UL, 0x3e4ccab7UL, 0x00000000UL, 0x3faf9c56UL, 0x1777f657UL, + 0x3e552f65UL, 0x00000000UL, 0x3fae91d2UL, 0xc317b86aUL, 0xbe5a61e0UL, + 0x00000000UL, 0x3fad87acUL, 0xb7664efbUL, 0xbe41f64eUL, 0x00000000UL, + 0x3fac7de6UL, 0x5d3d03a9UL, 0x3e0807a0UL, 0x00000000UL, 0x3fab7480UL, + 0x743c38ebUL, 0xbe3726e1UL, 0x00000000UL, 0x3faa6b78UL, 0x06a253f1UL, + 0x3e5ad636UL, 0x00000000UL, 0x3fa962d0UL, 0xa35f541bUL, 0x3e5a187aUL, + 0x00000000UL, 0x3fa85a88UL, 0x4b86e446UL, 0xbe508150UL, 0x00000000UL, + 0x3fa7529cUL, 0x2589cacfUL, 0x3e52938aUL, 0x00000000UL, 0x3fa64b10UL, + 0xaf6b11f2UL, 0xbe3454cdUL, 0x00000000UL, 0x3fa543e2UL, 0x97506fefUL, + 0xbe5fdec5UL, 0x00000000UL, 0x3fa43d10UL, 0xe75f7dd9UL, 0xbe388dd3UL, + 0x00000000UL, 0x3fa3369cUL, 0xa4139632UL, 0xbdea5177UL, 0x00000000UL, + 0x3fa23086UL, 0x352d6f1eUL, 0xbe565ad6UL, 0x00000000UL, 0x3fa12accUL, + 0x77449eb7UL, 0xbe50d5c7UL, 0x00000000UL, 0x3fa0256eUL, 0x7478da78UL, + 0x3e404724UL, 0x00000000UL, 0x3f9e40dcUL, 0xf59cef7fUL, 0xbe539d0aUL, + 0x00000000UL, 0x3f9c3790UL, 0x1511d43cUL, 0x3e53c2c8UL, 0x00000000UL, + 0x3f9a2f00UL, 0x9b8bff3cUL, 0xbe43b3e1UL, 0x00000000UL, 0x3f982724UL, + 0xad1e22a5UL, 0x3e46f0bdUL, 0x00000000UL, 0x3f962000UL, 0x130d9356UL, + 0x3e475ba0UL, 0x00000000UL, 0x3f941994UL, 0x8f86f883UL, 0xbe513d0bUL, + 0x00000000UL, 0x3f9213dcUL, 0x914d0dc8UL, 0xbe534335UL, 0x00000000UL, + 0x3f900ed8UL, 0x2d73e5e7UL, 0xbe22ba75UL, 0x00000000UL, 0x3f8c1510UL, + 0xc5b7d70eUL, 0x3e599c5dUL, 0x00000000UL, 0x3f880de0UL, 0x8a27857eUL, + 0xbe3d28c8UL, 0x00000000UL, 0x3f840810UL, 0xda767328UL, 0x3e531b3dUL, + 0x00000000UL, 0x3f8003b0UL, 0x77bacaf3UL, 0xbe5f04e3UL, 0x00000000UL, + 0x3f780150UL, 0xdf4b0720UL, 0x3e5a8bffUL, 0x00000000UL, 0x3f6ffc40UL, + 0x34c48e71UL, 0xbe3fcd99UL, 0x00000000UL, 0x3f5ff6c0UL, 0x1ad218afUL, + 0xbe4c78a7UL, 0x00000000UL, 0x00000000UL, 0x00000000UL, 0x80000000UL, + 0x00000000UL, 0xfffff800UL, 0x00000000UL, 0xfffff800UL, 0x00000000UL, + 0x3ff72000UL, 0x161bb241UL, 0xbf5dabe1UL, 0x6dc96112UL, 0xbf836578UL, + 0xee241472UL, 0xbf9b0301UL, 0x9f95985aUL, 0xbfb528dbUL, 0xb3841d2aUL, + 0xbfd619b6UL, 0x518775e3UL, 0x3f9004f2UL, 0xac8349bbUL, 0x3fa76c9bUL, + 0x486ececcUL, 0x3fc4635eUL, 0x161bb241UL, 0xbf5dabe1UL, 0x9f95985aUL, + 0xbfb528dbUL, 0xf8b5787dUL, 0x3ef2531eUL, 0x486ececbUL, 0x3fc4635eUL, + 0x412055ccUL, 0xbdd61bb2UL, 0x00000000UL, 0xfffffff8UL, 0x00000000UL, + 0xffffffffUL, 0x00000000UL, 0x3ff00000UL, 0x00000000UL, 0x3b700000UL, + 0xfa5abcbfUL, 0x3ff00b1aUL, 0xa7609f71UL, 0xbc84f6b2UL, 0xa9fb3335UL, + 0x3ff0163dUL, 0x9ab8cdb7UL, 0x3c9b6129UL, 0x143b0281UL, 0x3ff02168UL, + 0x0fc54eb6UL, 0xbc82bf31UL, 0x3e778061UL, 0x3ff02c9aUL, 0x535b085dUL, + 0xbc719083UL, 0x2e11bbccUL, 0x3ff037d4UL, 0xeeade11aUL, 0x3c656811UL, + 0xe86e7f85UL, 0x3ff04315UL, 0x1977c96eUL, 0xbc90a31cUL, 0x72f654b1UL, + 0x3ff04e5fUL, 0x3aa0d08cUL, 0x3c84c379UL, 0xd3158574UL, 0x3ff059b0UL, + 0xa475b465UL, 0x3c8d73e2UL, 0x0e3c1f89UL, 0x3ff0650aUL, 0x5799c397UL, + 0xbc95cb7bUL, 0x29ddf6deUL, 0x3ff0706bUL, 0xe2b13c27UL, 0xbc8c91dfUL, + 0x2b72a836UL, 0x3ff07bd4UL, 0x54458700UL, 0x3c832334UL, 0x18759bc8UL, + 0x3ff08745UL, 0x4bb284ffUL, 0x3c6186beUL, 0xf66607e0UL, 0x3ff092bdUL, + 0x800a3fd1UL, 0xbc968063UL, 0xcac6f383UL, 0x3ff09e3eUL, 0x18316136UL, + 0x3c914878UL, 0x9b1f3919UL, 0x3ff0a9c7UL, 0x873d1d38UL, 0x3c85d16cUL, + 0x6cf9890fUL, 0x3ff0b558UL, 0x4adc610bUL, 0x3c98a62eUL, 0x45e46c85UL, + 0x3ff0c0f1UL, 0x06d21cefUL, 0x3c94f989UL, 0x2b7247f7UL, 0x3ff0cc92UL, + 0x16e24f71UL, 0x3c901edcUL, 0x23395decUL, 0x3ff0d83bUL, 0xe43f316aUL, + 0xbc9bc14dUL, 0x32d3d1a2UL, 0x3ff0e3ecUL, 0x27c57b52UL, 0x3c403a17UL, + 0x5fdfa9c5UL, 0x3ff0efa5UL, 0xbc54021bUL, 0xbc949db9UL, 0xaffed31bUL, + 0x3ff0fb66UL, 0xc44ebd7bUL, 0xbc6b9bedUL, 0x28d7233eUL, 0x3ff10730UL, + 0x1692fdd5UL, 0x3c8d46ebUL, 0xd0125b51UL, 0x3ff11301UL, 0x39449b3aUL, + 0xbc96c510UL, 0xab5e2ab6UL, 0x3ff11edbUL, 0xf703fb72UL, 0xbc9ca454UL, + 0xc06c31ccUL, 0x3ff12abdUL, 0xb36ca5c7UL, 0xbc51b514UL, 0x14f204abUL, + 0x3ff136a8UL, 0xba48dcf0UL, 0xbc67108fUL, 0xaea92de0UL, 0x3ff1429aUL, + 0x9af1369eUL, 0xbc932fbfUL, 0x934f312eUL, 0x3ff14e95UL, 0x39bf44abUL, + 0xbc8b91e8UL, 0xc8a58e51UL, 0x3ff15a98UL, 0xb9eeab0aUL, 0x3c82406aUL, + 0x5471c3c2UL, 0x3ff166a4UL, 0x82ea1a32UL, 0x3c58f23bUL, 0x3c7d517bUL, + 0x3ff172b8UL, 0xb9d78a76UL, 0xbc819041UL, 0x8695bbc0UL, 0x3ff17ed4UL, + 0xe2ac5a64UL, 0x3c709e3fUL, 0x388c8deaUL, 0x3ff18af9UL, 0xd1970f6cUL, + 0xbc911023UL, 0x58375d2fUL, 0x3ff19726UL, 0x85f17e08UL, 0x3c94aaddUL, + 0xeb6fcb75UL, 0x3ff1a35bUL, 0x7b4968e4UL, 0x3c8e5b4cUL, 0xf8138a1cUL, + 0x3ff1af99UL, 0xa4b69280UL, 0x3c97bf85UL, 0x84045cd4UL, 0x3ff1bbe0UL, + 0x352ef607UL, 0xbc995386UL, 0x95281c6bUL, 0x3ff1c82fUL, 0x8010f8c9UL, + 0x3c900977UL, 0x3168b9aaUL, 0x3ff1d487UL, 0x00a2643cUL, 0x3c9e016eUL, + 0x5eb44027UL, 0x3ff1e0e7UL, 0x088cb6deUL, 0xbc96fdd8UL, 0x22fcd91dUL, + 0x3ff1ed50UL, 0x027bb78cUL, 0xbc91df98UL, 0x8438ce4dUL, 0x3ff1f9c1UL, + 0xa097af5cUL, 0xbc9bf524UL, 0x88628cd6UL, 0x3ff2063bUL, 0x814a8495UL, + 0x3c8dc775UL, 0x3578a819UL, 0x3ff212beUL, 0x2cfcaac9UL, 0x3c93592dUL, + 0x917ddc96UL, 0x3ff21f49UL, 0x9494a5eeUL, 0x3c82a97eUL, 0xa27912d1UL, + 0x3ff22bddUL, 0x5577d69fUL, 0x3c8d34fbUL, 0x6e756238UL, 0x3ff2387aUL, + 0xb6c70573UL, 0x3c99b07eUL, 0xfb82140aUL, 0x3ff2451fUL, 0x911ca996UL, + 0x3c8acfccUL, 0x4fb2a63fUL, 0x3ff251ceUL, 0xbef4f4a4UL, 0x3c8ac155UL, + 0x711ece75UL, 0x3ff25e85UL, 0x4ac31b2cUL, 0x3c93e1a2UL, 0x65e27cddUL, + 0x3ff26b45UL, 0x9940e9d9UL, 0x3c82bd33UL, 0x341ddf29UL, 0x3ff2780eUL, + 0x05f9e76cUL, 0x3c9e067cUL, 0xe1f56381UL, 0x3ff284dfUL, 0x8c3f0d7eUL, + 0xbc9a4c3aUL, 0x7591bb70UL, 0x3ff291baUL, 0x28401cbdUL, 0xbc82cc72UL, + 0xf51fdee1UL, 0x3ff29e9dUL, 0xafad1255UL, 0x3c8612e8UL, 0x66d10f13UL, + 0x3ff2ab8aUL, 0x191690a7UL, 0xbc995743UL, 0xd0dad990UL, 0x3ff2b87fUL, + 0xd6381aa4UL, 0xbc410adcUL, 0x39771b2fUL, 0x3ff2c57eUL, 0xa6eb5124UL, + 0xbc950145UL, 0xa6e4030bUL, 0x3ff2d285UL, 0x54db41d5UL, 0x3c900247UL, + 0x1f641589UL, 0x3ff2df96UL, 0xfbbce198UL, 0x3c9d16cfUL, 0xa93e2f56UL, + 0x3ff2ecafUL, 0x45d52383UL, 0x3c71ca0fUL, 0x4abd886bUL, 0x3ff2f9d2UL, + 0x532bda93UL, 0xbc653c55UL, 0x0a31b715UL, 0x3ff306feUL, 0xd23182e4UL, + 0x3c86f46aUL, 0xedeeb2fdUL, 0x3ff31432UL, 0xf3f3fcd1UL, 0x3c8959a3UL, + 0xfc4cd831UL, 0x3ff32170UL, 0x8e18047cUL, 0x3c8a9ce7UL, 0x3ba8ea32UL, + 0x3ff32eb8UL, 0x3cb4f318UL, 0xbc9c45e8UL, 0xb26416ffUL, 0x3ff33c08UL, + 0x843659a6UL, 0x3c932721UL, 0x66e3fa2dUL, 0x3ff34962UL, 0x930881a4UL, + 0xbc835a75UL, 0x5f929ff1UL, 0x3ff356c5UL, 0x5c4e4628UL, 0xbc8b5ceeUL, + 0xa2de883bUL, 0x3ff36431UL, 0xa06cb85eUL, 0xbc8c3144UL, 0x373aa9cbUL, + 0x3ff371a7UL, 0xbf42eae2UL, 0xbc963aeaUL, 0x231e754aUL, 0x3ff37f26UL, + 0x9eceb23cUL, 0xbc99f5caUL, 0x6d05d866UL, 0x3ff38caeUL, 0x3c9904bdUL, + 0xbc9e958dUL, 0x1b7140efUL, 0x3ff39a40UL, 0xfc8e2934UL, 0xbc99a9a5UL, + 0x34e59ff7UL, 0x3ff3a7dbUL, 0xd661f5e3UL, 0xbc75e436UL, 0xbfec6cf4UL, + 0x3ff3b57fUL, 0xe26fff18UL, 0x3c954c66UL, 0xc313a8e5UL, 0x3ff3c32dUL, + 0x375d29c3UL, 0xbc9efff8UL, 0x44ede173UL, 0x3ff3d0e5UL, 0x8c284c71UL, + 0x3c7fe8d0UL, 0x4c123422UL, 0x3ff3dea6UL, 0x11f09ebcUL, 0x3c8ada09UL, + 0xdf1c5175UL, 0x3ff3ec70UL, 0x7b8c9bcaUL, 0xbc8af663UL, 0x04ac801cUL, + 0x3ff3fa45UL, 0xf956f9f3UL, 0xbc97d023UL, 0xc367a024UL, 0x3ff40822UL, + 0xb6f4d048UL, 0x3c8bddf8UL, 0x21f72e2aUL, 0x3ff4160aUL, 0x1c309278UL, + 0xbc5ef369UL, 0x2709468aUL, 0x3ff423fbUL, 0xc0b314ddUL, 0xbc98462dUL, + 0xd950a897UL, 0x3ff431f5UL, 0xe35f7999UL, 0xbc81c7ddUL, 0x3f84b9d4UL, + 0x3ff43ffaUL, 0x9704c003UL, 0x3c8880beUL, 0x6061892dUL, 0x3ff44e08UL, + 0x04ef80d0UL, 0x3c489b7aUL, 0x42a7d232UL, 0x3ff45c20UL, 0x82fb1f8eUL, + 0xbc686419UL, 0xed1d0057UL, 0x3ff46a41UL, 0xd1648a76UL, 0x3c9c944bUL, + 0x668b3237UL, 0x3ff4786dUL, 0xed445733UL, 0xbc9c20f0UL, 0xb5c13cd0UL, + 0x3ff486a2UL, 0xb69062f0UL, 0x3c73c1a3UL, 0xe192aed2UL, 0x3ff494e1UL, + 0x5e499ea0UL, 0xbc83b289UL, 0xf0d7d3deUL, 0x3ff4a32aUL, 0xf3d1be56UL, + 0x3c99cb62UL, 0xea6db7d7UL, 0x3ff4b17dUL, 0x7f2897f0UL, 0xbc8125b8UL, + 0xd5362a27UL, 0x3ff4bfdaUL, 0xafec42e2UL, 0x3c7d4397UL, 0xb817c114UL, + 0x3ff4ce41UL, 0x690abd5dUL, 0x3c905e29UL, 0x99fddd0dUL, 0x3ff4dcb2UL, + 0xbc6a7833UL, 0x3c98ecdbUL, 0x81d8abffUL, 0x3ff4eb2dUL, 0x2e5d7a52UL, + 0xbc95257dUL, 0x769d2ca7UL, 0x3ff4f9b2UL, 0xd25957e3UL, 0xbc94b309UL, + 0x7f4531eeUL, 0x3ff50841UL, 0x49b7465fUL, 0x3c7a249bUL, 0xa2cf6642UL, + 0x3ff516daUL, 0x69bd93efUL, 0xbc8f7685UL, 0xe83f4eefUL, 0x3ff5257dUL, + 0x43efef71UL, 0xbc7c998dUL, 0x569d4f82UL, 0x3ff5342bUL, 0x1db13cadUL, + 0xbc807abeUL, 0xf4f6ad27UL, 0x3ff542e2UL, 0x192d5f7eUL, 0x3c87926dUL, + 0xca5d920fUL, 0x3ff551a4UL, 0xefede59bUL, 0xbc8d689cUL, 0xdde910d2UL, + 0x3ff56070UL, 0x168eebf0UL, 0xbc90fb6eUL, 0x36b527daUL, 0x3ff56f47UL, + 0x011d93adUL, 0x3c99bb2cUL, 0xdbe2c4cfUL, 0x3ff57e27UL, 0x8a57b9c4UL, + 0xbc90b98cUL, 0xd497c7fdUL, 0x3ff58d12UL, 0x5b9a1de8UL, 0x3c8295e1UL, + 0x27ff07ccUL, 0x3ff59c08UL, 0xe467e60fUL, 0xbc97e2ceUL, 0xdd485429UL, + 0x3ff5ab07UL, 0x054647adUL, 0x3c96324cUL, 0xfba87a03UL, 0x3ff5ba11UL, + 0x4c233e1aUL, 0xbc9b77a1UL, 0x8a5946b7UL, 0x3ff5c926UL, 0x816986a2UL, + 0x3c3c4b1bUL, 0x90998b93UL, 0x3ff5d845UL, 0xa8b45643UL, 0xbc9cd6a7UL, + 0x15ad2148UL, 0x3ff5e76fUL, 0x3080e65eUL, 0x3c9ba6f9UL, 0x20dceb71UL, + 0x3ff5f6a3UL, 0xe3cdcf92UL, 0xbc89eaddUL, 0xb976dc09UL, 0x3ff605e1UL, + 0x9b56de47UL, 0xbc93e242UL, 0xe6cdf6f4UL, 0x3ff6152aUL, 0x4ab84c27UL, + 0x3c9e4b3eUL, 0xb03a5585UL, 0x3ff6247eUL, 0x7e40b497UL, 0xbc9383c1UL, + 0x1d1929fdUL, 0x3ff633ddUL, 0xbeb964e5UL, 0x3c984710UL, 0x34ccc320UL, + 0x3ff64346UL, 0x759d8933UL, 0xbc8c483cUL, 0xfebc8fb7UL, 0x3ff652b9UL, + 0xc9a73e09UL, 0xbc9ae3d5UL, 0x82552225UL, 0x3ff66238UL, 0x87591c34UL, + 0xbc9bb609UL, 0xc70833f6UL, 0x3ff671c1UL, 0x586c6134UL, 0xbc8e8732UL, + 0xd44ca973UL, 0x3ff68155UL, 0x44f73e65UL, 0x3c6038aeUL, 0xb19e9538UL, + 0x3ff690f4UL, 0x9aeb445dUL, 0x3c8804bdUL, 0x667f3bcdUL, 0x3ff6a09eUL, + 0x13b26456UL, 0xbc9bdd34UL, 0xfa75173eUL, 0x3ff6b052UL, 0x2c9a9d0eUL, + 0x3c7a38f5UL, 0x750bdabfUL, 0x3ff6c012UL, 0x67ff0b0dUL, 0xbc728956UL, + 0xddd47645UL, 0x3ff6cfdcUL, 0xb6f17309UL, 0x3c9c7aa9UL, 0x3c651a2fUL, + 0x3ff6dfb2UL, 0x683c88abUL, 0xbc6bbe3aUL, 0x98593ae5UL, 0x3ff6ef92UL, + 0x9e1ac8b2UL, 0xbc90b974UL, 0xf9519484UL, 0x3ff6ff7dUL, 0x25860ef6UL, + 0xbc883c0fUL, 0x66f42e87UL, 0x3ff70f74UL, 0xd45aa65fUL, 0x3c59d644UL, + 0xe8ec5f74UL, 0x3ff71f75UL, 0x86887a99UL, 0xbc816e47UL, 0x86ead08aUL, + 0x3ff72f82UL, 0x2cd62c72UL, 0xbc920aa0UL, 0x48a58174UL, 0x3ff73f9aUL, + 0x6c65d53cUL, 0xbc90a8d9UL, 0x35d7cbfdUL, 0x3ff74fbdUL, 0x618a6e1cUL, + 0x3c9047fdUL, 0x564267c9UL, 0x3ff75febUL, 0x57316dd3UL, 0xbc902459UL, + 0xb1ab6e09UL, 0x3ff77024UL, 0x169147f8UL, 0x3c9b7877UL, 0x4fde5d3fUL, + 0x3ff78069UL, 0x0a02162dUL, 0x3c9866b8UL, 0x38ac1cf6UL, 0x3ff790b9UL, + 0x62aadd3eUL, 0x3c9349a8UL, 0x73eb0187UL, 0x3ff7a114UL, 0xee04992fUL, + 0xbc841577UL, 0x0976cfdbUL, 0x3ff7b17bUL, 0x8468dc88UL, 0xbc9bebb5UL, + 0x0130c132UL, 0x3ff7c1edUL, 0xd1164dd6UL, 0x3c9f124cUL, 0x62ff86f0UL, + 0x3ff7d26aUL, 0xfb72b8b4UL, 0x3c91bddbUL, 0x36cf4e62UL, 0x3ff7e2f3UL, + 0xba15797eUL, 0x3c705d02UL, 0x8491c491UL, 0x3ff7f387UL, 0xcf9311aeUL, + 0xbc807f11UL, 0x543e1a12UL, 0x3ff80427UL, 0x626d972bUL, 0xbc927c86UL, + 0xadd106d9UL, 0x3ff814d2UL, 0x0d151d4dUL, 0x3c946437UL, 0x994cce13UL, + 0x3ff82589UL, 0xd41532d8UL, 0xbc9d4c1dUL, 0x1eb941f7UL, 0x3ff8364cUL, + 0x31df2bd5UL, 0x3c999b9aUL, 0x4623c7adUL, 0x3ff8471aUL, 0xa341cdfbUL, + 0xbc88d684UL, 0x179f5b21UL, 0x3ff857f4UL, 0xf8b216d0UL, 0xbc5ba748UL, + 0x9b4492edUL, 0x3ff868d9UL, 0x9bd4f6baUL, 0xbc9fc6f8UL, 0xd931a436UL, + 0x3ff879caUL, 0xd2db47bdUL, 0x3c85d2d7UL, 0xd98a6699UL, 0x3ff88ac7UL, + 0xf37cb53aUL, 0x3c9994c2UL, 0xa478580fUL, 0x3ff89bd0UL, 0x4475202aUL, + 0x3c9d5395UL, 0x422aa0dbUL, 0x3ff8ace5UL, 0x56864b27UL, 0x3c96e9f1UL, + 0xbad61778UL, 0x3ff8be05UL, 0xfc43446eUL, 0x3c9ecb5eUL, 0x16b5448cUL, + 0x3ff8cf32UL, 0x32e9e3aaUL, 0xbc70d55eUL, 0x5e0866d9UL, 0x3ff8e06aUL, + 0x6fc9b2e6UL, 0xbc97114aUL, 0x99157736UL, 0x3ff8f1aeUL, 0xa2e3976cUL, + 0x3c85cc13UL, 0xd0282c8aUL, 0x3ff902feUL, 0x85fe3fd2UL, 0x3c9592caUL, + 0x0b91ffc6UL, 0x3ff9145bUL, 0x2e582524UL, 0xbc9dd679UL, 0x53aa2fe2UL, + 0x3ff925c3UL, 0xa639db7fUL, 0xbc83455fUL, 0xb0cdc5e5UL, 0x3ff93737UL, + 0x81b57ebcUL, 0xbc675fc7UL, 0x2b5f98e5UL, 0x3ff948b8UL, 0x797d2d99UL, + 0xbc8dc3d6UL, 0xcbc8520fUL, 0x3ff95a44UL, 0x96a5f039UL, 0xbc764b7cUL, + 0x9a7670b3UL, 0x3ff96bddUL, 0x7f19c896UL, 0xbc5ba596UL, 0x9fde4e50UL, + 0x3ff97d82UL, 0x7c1b85d1UL, 0xbc9d185bUL, 0xe47a22a2UL, 0x3ff98f33UL, + 0xa24c78ecUL, 0x3c7cabdaUL, 0x70ca07baUL, 0x3ff9a0f1UL, 0x91cee632UL, + 0xbc9173bdUL, 0x4d53fe0dUL, 0x3ff9b2bbUL, 0x4df6d518UL, 0xbc9dd84eUL, + 0x82a3f090UL, 0x3ff9c491UL, 0xb071f2beUL, 0x3c7c7c46UL, 0x194bb8d5UL, + 0x3ff9d674UL, 0xa3dd8233UL, 0xbc9516beUL, 0x19e32323UL, 0x3ff9e863UL, + 0x78e64c6eUL, 0x3c7824caUL, 0x8d07f29eUL, 0x3ff9fa5eUL, 0xaaf1faceUL, + 0xbc84a9ceUL, 0x7b5de565UL, 0x3ffa0c66UL, 0x5d1cd533UL, 0xbc935949UL, + 0xed8eb8bbUL, 0x3ffa1e7aUL, 0xee8be70eUL, 0x3c9c6618UL, 0xec4a2d33UL, + 0x3ffa309bUL, 0x7ddc36abUL, 0x3c96305cUL, 0x80460ad8UL, 0x3ffa42c9UL, + 0x589fb120UL, 0xbc9aa780UL, 0xb23e255dUL, 0x3ffa5503UL, 0xdb8d41e1UL, + 0xbc9d2f6eUL, 0x8af46052UL, 0x3ffa674aUL, 0x30670366UL, 0x3c650f56UL, + 0x1330b358UL, 0x3ffa799eUL, 0xcac563c7UL, 0x3c9bcb7eUL, 0x53c12e59UL, + 0x3ffa8bfeUL, 0xb2ba15a9UL, 0xbc94f867UL, 0x5579fdbfUL, 0x3ffa9e6bUL, + 0x0ef7fd31UL, 0x3c90fac9UL, 0x21356ebaUL, 0x3ffab0e5UL, 0xdae94545UL, + 0x3c889c31UL, 0xbfd3f37aUL, 0x3ffac36bUL, 0xcae76cd0UL, 0xbc8f9234UL, + 0x3a3c2774UL, 0x3ffad5ffUL, 0xb6b1b8e5UL, 0x3c97ef3bUL, 0x995ad3adUL, + 0x3ffae89fUL, 0x345dcc81UL, 0x3c97a1cdUL, 0xe622f2ffUL, 0x3ffafb4cUL, + 0x0f315ecdUL, 0xbc94b2fcUL, 0x298db666UL, 0x3ffb0e07UL, 0x4c80e425UL, + 0xbc9bdef5UL, 0x6c9a8952UL, 0x3ffb20ceUL, 0x4a0756ccUL, 0x3c94dd02UL, + 0xb84f15fbUL, 0x3ffb33a2UL, 0x3084d708UL, 0xbc62805eUL, 0x15b749b1UL, + 0x3ffb4684UL, 0xe9df7c90UL, 0xbc7f763dUL, 0x8de5593aUL, 0x3ffb5972UL, + 0xbbba6de3UL, 0xbc9c71dfUL, 0x29f1c52aUL, 0x3ffb6c6eUL, 0x52883f6eUL, + 0x3c92a8f3UL, 0xf2fb5e47UL, 0x3ffb7f76UL, 0x7e54ac3bUL, 0xbc75584fUL, + 0xf22749e4UL, 0x3ffb928cUL, 0x54cb65c6UL, 0xbc9b7216UL, 0x30a1064aUL, + 0x3ffba5b0UL, 0x0e54292eUL, 0xbc9efcd3UL, 0xb79a6f1fUL, 0x3ffbb8e0UL, + 0xc9696205UL, 0xbc3f52d1UL, 0x904bc1d2UL, 0x3ffbcc1eUL, 0x7a2d9e84UL, + 0x3c823dd0UL, 0xc3f3a207UL, 0x3ffbdf69UL, 0x60ea5b53UL, 0xbc3c2623UL, + 0x5bd71e09UL, 0x3ffbf2c2UL, 0x3f6b9c73UL, 0xbc9efdcaUL, 0x6141b33dUL, + 0x3ffc0628UL, 0xa1fbca34UL, 0xbc8d8a5aUL, 0xdd85529cUL, 0x3ffc199bUL, + 0x895048ddUL, 0x3c811065UL, 0xd9fa652cUL, 0x3ffc2d1cUL, 0x17c8a5d7UL, + 0xbc96e516UL, 0x5fffd07aUL, 0x3ffc40abUL, 0xe083c60aUL, 0x3c9b4537UL, + 0x78fafb22UL, 0x3ffc5447UL, 0x2493b5afUL, 0x3c912f07UL, 0x2e57d14bUL, + 0x3ffc67f1UL, 0xff483cadUL, 0x3c92884dUL, 0x8988c933UL, 0x3ffc7ba8UL, + 0xbe255559UL, 0xbc8e76bbUL, 0x9406e7b5UL, 0x3ffc8f6dUL, 0x48805c44UL, + 0x3c71acbcUL, 0x5751c4dbUL, 0x3ffca340UL, 0xd10d08f5UL, 0xbc87f2beUL, + 0xdcef9069UL, 0x3ffcb720UL, 0xd1e949dbUL, 0x3c7503cbUL, 0x2e6d1675UL, + 0x3ffccb0fUL, 0x86009092UL, 0xbc7d220fUL, 0x555dc3faUL, 0x3ffcdf0bUL, + 0x53829d72UL, 0xbc8dd83bUL, 0x5b5bab74UL, 0x3ffcf315UL, 0xb86dff57UL, + 0xbc9a08e9UL, 0x4a07897cUL, 0x3ffd072dUL, 0x43797a9cUL, 0xbc9cbc37UL, + 0x2b08c968UL, 0x3ffd1b53UL, 0x219a36eeUL, 0x3c955636UL, 0x080d89f2UL, + 0x3ffd2f87UL, 0x719d8578UL, 0xbc9d487bUL, 0xeacaa1d6UL, 0x3ffd43c8UL, + 0xbf5a1614UL, 0x3c93db53UL, 0xdcfba487UL, 0x3ffd5818UL, 0xd75b3707UL, + 0x3c82ed02UL, 0xe862e6d3UL, 0x3ffd6c76UL, 0x4a8165a0UL, 0x3c5fe87aUL, + 0x16c98398UL, 0x3ffd80e3UL, 0x8beddfe8UL, 0xbc911ec1UL, 0x71ff6075UL, + 0x3ffd955dUL, 0xbb9af6beUL, 0x3c9a052dUL, 0x03db3285UL, 0x3ffda9e6UL, + 0x696db532UL, 0x3c9c2300UL, 0xd63a8315UL, 0x3ffdbe7cUL, 0x926b8be4UL, + 0xbc9b76f1UL, 0xf301b460UL, 0x3ffdd321UL, 0x78f018c3UL, 0x3c92da57UL, + 0x641c0658UL, 0x3ffde7d5UL, 0x8e79ba8fUL, 0xbc9ca552UL, 0x337b9b5fUL, + 0x3ffdfc97UL, 0x4f184b5cUL, 0xbc91a5cdUL, 0x6b197d17UL, 0x3ffe1167UL, + 0xbd5c7f44UL, 0xbc72b529UL, 0x14f5a129UL, 0x3ffe2646UL, 0x817a1496UL, + 0xbc97b627UL, 0x3b16ee12UL, 0x3ffe3b33UL, 0x31fdc68bUL, 0xbc99f4a4UL, + 0xe78b3ff6UL, 0x3ffe502eUL, 0x80a9cc8fUL, 0x3c839e89UL, 0x24676d76UL, + 0x3ffe6539UL, 0x7522b735UL, 0xbc863ff8UL, 0xfbc74c83UL, 0x3ffe7a51UL, + 0xca0c8de2UL, 0x3c92d522UL, 0x77cdb740UL, 0x3ffe8f79UL, 0x80b054b1UL, + 0xbc910894UL, 0xa2a490daUL, 0x3ffea4afUL, 0x179c2893UL, 0xbc9e9c23UL, + 0x867cca6eUL, 0x3ffeb9f4UL, 0x2293e4f2UL, 0x3c94832fUL, 0x2d8e67f1UL, + 0x3ffecf48UL, 0xb411ad8cUL, 0xbc9c93f3UL, 0xa2188510UL, 0x3ffee4aaUL, + 0xa487568dUL, 0x3c91c68dUL, 0xee615a27UL, 0x3ffefa1bUL, 0x86a4b6b0UL, + 0x3c9dc7f4UL, 0x1cb6412aUL, 0x3fff0f9cUL, 0x65181d45UL, 0xbc932200UL, + 0x376bba97UL, 0x3fff252bUL, 0xbf0d8e43UL, 0x3c93a1a5UL, 0x48dd7274UL, + 0x3fff3ac9UL, 0x3ed837deUL, 0xbc795a5aUL, 0x5b6e4540UL, 0x3fff5076UL, + 0x2dd8a18bUL, 0x3c99d3e1UL, 0x798844f8UL, 0x3fff6632UL, 0x3539343eUL, + 0x3c9fa37bUL, 0xad9cbe14UL, 0x3fff7bfdUL, 0xd006350aUL, 0xbc9dbb12UL, + 0x02243c89UL, 0x3fff91d8UL, 0xa779f689UL, 0xbc612ea8UL, 0x819e90d8UL, + 0x3fffa7c1UL, 0xf3a5931eUL, 0x3c874853UL, 0x3692d514UL, 0x3fffbdbaUL, + 0x15098eb6UL, 0xbc796773UL, 0x2b8f71f1UL, 0x3fffd3c2UL, 0x966579e7UL, + 0x3c62eb74UL, 0x6b2a23d9UL, 0x3fffe9d9UL, 0x7442fde3UL, 0x3c74a603UL, + 0xe78a6731UL, 0x3f55d87fUL, 0xd704a0c0UL, 0x3fac6b08UL, 0x6fba4e77UL, + 0x3f83b2abUL, 0xff82c58fUL, 0x3fcebfbdUL, 0xfefa39efUL, 0x3fe62e42UL, + 0x00000000UL, 0x00000000UL, 0xfefa39efUL, 0x3fe62e42UL, 0xfefa39efUL, + 0xbfe62e42UL, 0xf8000000UL, 0xffffffffUL, 0xf8000000UL, 0xffffffffUL, + 0x00000000UL, 0x80000000UL, 0x00000000UL, 0x00000000UL + +}; + +//registers, +// input: xmm0, xmm1 +// scratch: xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7 +// eax, edx, ecx, ebx + +// Code generated by Intel C compiler for LIBM library + +void MacroAssembler::fast_pow(XMMRegister xmm0, XMMRegister xmm1, XMMRegister xmm2, XMMRegister xmm3, XMMRegister xmm4, XMMRegister xmm5, XMMRegister xmm6, XMMRegister xmm7, Register eax, Register ecx, Register edx, Register tmp) { + Label L_2TAG_PACKET_0_0_2, L_2TAG_PACKET_1_0_2, L_2TAG_PACKET_2_0_2, L_2TAG_PACKET_3_0_2; + Label L_2TAG_PACKET_4_0_2, L_2TAG_PACKET_5_0_2, L_2TAG_PACKET_6_0_2, L_2TAG_PACKET_7_0_2; + Label L_2TAG_PACKET_8_0_2, L_2TAG_PACKET_9_0_2, L_2TAG_PACKET_10_0_2, L_2TAG_PACKET_11_0_2; + Label L_2TAG_PACKET_12_0_2, L_2TAG_PACKET_13_0_2, L_2TAG_PACKET_14_0_2, L_2TAG_PACKET_15_0_2; + Label L_2TAG_PACKET_16_0_2, L_2TAG_PACKET_17_0_2, L_2TAG_PACKET_18_0_2, L_2TAG_PACKET_19_0_2; + Label L_2TAG_PACKET_20_0_2, L_2TAG_PACKET_21_0_2, L_2TAG_PACKET_22_0_2, L_2TAG_PACKET_23_0_2; + Label L_2TAG_PACKET_24_0_2, L_2TAG_PACKET_25_0_2, L_2TAG_PACKET_26_0_2, L_2TAG_PACKET_27_0_2; + Label L_2TAG_PACKET_28_0_2, L_2TAG_PACKET_29_0_2, L_2TAG_PACKET_30_0_2, L_2TAG_PACKET_31_0_2; + Label L_2TAG_PACKET_32_0_2, L_2TAG_PACKET_33_0_2, L_2TAG_PACKET_34_0_2, L_2TAG_PACKET_35_0_2; + Label L_2TAG_PACKET_36_0_2, L_2TAG_PACKET_37_0_2, L_2TAG_PACKET_38_0_2, L_2TAG_PACKET_39_0_2; + Label L_2TAG_PACKET_40_0_2, L_2TAG_PACKET_41_0_2, L_2TAG_PACKET_42_0_2, L_2TAG_PACKET_43_0_2; + Label L_2TAG_PACKET_44_0_2, L_2TAG_PACKET_45_0_2, L_2TAG_PACKET_46_0_2, L_2TAG_PACKET_47_0_2; + Label L_2TAG_PACKET_48_0_2, L_2TAG_PACKET_49_0_2, L_2TAG_PACKET_50_0_2, L_2TAG_PACKET_51_0_2; + Label L_2TAG_PACKET_52_0_2, L_2TAG_PACKET_53_0_2, L_2TAG_PACKET_54_0_2, L_2TAG_PACKET_55_0_2; + Label L_2TAG_PACKET_56_0_2, L_2TAG_PACKET_57_0_2, L_2TAG_PACKET_58_0_2, start; + + assert_different_registers(tmp, eax, ecx, edx); + + address static_const_table_pow = (address)_static_const_table_pow; + + bind(start); + subl(rsp, 120); + movl(Address(rsp, 64), tmp); + lea(tmp, ExternalAddress(static_const_table_pow)); + movsd(xmm0, Address(rsp, 128)); + movsd(xmm1, Address(rsp, 136)); + xorpd(xmm2, xmm2); + movl(eax, 16368); + pinsrw(xmm2, eax, 3); + movl(ecx, 1069088768); + movdl(xmm7, ecx); + movsd(Address(rsp, 16), xmm1); + xorpd(xmm1, xmm1); + movl(edx, 30704); + pinsrw(xmm1, edx, 3); + movsd(Address(rsp, 8), xmm0); + movdqu(xmm3, xmm0); + movl(edx, 8192); + movdl(xmm4, edx); + movdqu(xmm6, Address(tmp, 8240)); + pextrw(eax, xmm0, 3); + por(xmm0, xmm2); + psllq(xmm0, 5); + movsd(xmm2, Address(tmp, 8256)); + psrlq(xmm0, 34); + movl(edx, eax); + andl(edx, 32752); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + rcpss(xmm0, xmm0); + psllq(xmm3, 12); + addl(ecx, 16); + bsrl(ecx, ecx); + psrlq(xmm3, 12); + movl(Address(rsp, 24), rsi); + subl(eax, 16); + cmpl(eax, 32736); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_0_0_2); + movl(rsi, 0); + + bind(L_2TAG_PACKET_1_0_2); + mulss(xmm0, xmm7); + movl(edx, -1); + subl(ecx, 4); + shll(edx); + movdl(xmm5, edx); + por(xmm3, xmm1); + subl(eax, 16351); + cmpl(eax, 1); + jcc(Assembler::belowEqual, L_2TAG_PACKET_2_0_2); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + pand(xmm5, xmm3); + + bind(L_2TAG_PACKET_3_0_2); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + subl(eax, 1); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + + bind(L_2TAG_PACKET_4_0_2); + mulsd(xmm3, xmm0); + movdqu(xmm1, Address(tmp, 8272)); + subsd(xmm5, xmm2); + movdqu(xmm4, Address(tmp, 8288)); + movl(ecx, eax); + sarl(eax, 31); + addl(ecx, eax); + xorl(eax, ecx); + addl(eax, 1); + bsrl(eax, eax); + unpcklpd(xmm5, xmm3); + movdqu(xmm6, Address(tmp, 8304)); + addsd(xmm3, xmm5); + andl(edx, 16760832); + shrl(edx, 10); + addpd(xmm5, Address(tmp, edx, Address::times_1, -3616)); + movdqu(xmm0, Address(tmp, 8320)); + pshufd(xmm2, xmm3, 68); + mulsd(xmm3, xmm3); + mulpd(xmm1, xmm2); + mulpd(xmm4, xmm2); + addsd(xmm5, xmm7); + mulsd(xmm2, xmm3); + addpd(xmm6, xmm1); + mulsd(xmm3, xmm3); + addpd(xmm0, xmm4); + movsd(xmm1, Address(rsp, 16)); + movzwl(ecx, Address(rsp, 22)); + pshufd(xmm7, xmm5, 238); + movsd(xmm4, Address(tmp, 8368)); + mulpd(xmm6, xmm2); + pshufd(xmm3, xmm3, 68); + mulpd(xmm0, xmm2); + shll(eax, 4); + subl(eax, 15872); + andl(ecx, 32752); + addl(eax, ecx); + mulpd(xmm3, xmm6); + cmpl(eax, 624); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_5_0_2); + xorpd(xmm6, xmm6); + movl(edx, 17080); + pinsrw(xmm6, edx, 3); + movdqu(xmm2, xmm1); + pand(xmm4, xmm1); + subsd(xmm1, xmm4); + mulsd(xmm4, xmm5); + addsd(xmm0, xmm7); + mulsd(xmm1, xmm5); + movdqu(xmm7, xmm6); + addsd(xmm6, xmm4); + addpd(xmm3, xmm0); + movdl(edx, xmm6); + subsd(xmm6, xmm7); + pshufd(xmm0, xmm3, 238); + subsd(xmm4, xmm6); + addsd(xmm0, xmm3); + movl(ecx, edx); + andl(edx, 255); + addl(edx, edx); + movdqu(xmm5, Address(tmp, edx, Address::times_8, 8384)); + addsd(xmm4, xmm1); + mulsd(xmm2, xmm0); + movdqu(xmm7, Address(tmp, 12480)); + movdqu(xmm3, Address(tmp, 12496)); + shll(ecx, 12); + xorl(ecx, rsi); + andl(ecx, -1048576); + movdl(xmm6, ecx); + addsd(xmm2, xmm4); + movsd(xmm1, Address(tmp, 12512)); + pshufd(xmm0, xmm2, 68); + pshufd(xmm4, xmm2, 68); + mulpd(xmm0, xmm0); + movl(rsi, Address(rsp, 24)); + mulpd(xmm7, xmm4); + pshufd(xmm6, xmm6, 17); + mulsd(xmm1, xmm2); + mulsd(xmm0, xmm0); + paddd(xmm5, xmm6); + addpd(xmm3, xmm7); + mulsd(xmm1, xmm5); + pshufd(xmm6, xmm5, 238); + mulpd(xmm0, xmm3); + addsd(xmm1, xmm6); + pshufd(xmm3, xmm0, 238); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + addsd(xmm0, xmm1); + addsd(xmm0, xmm3); + addsd(xmm0, xmm5); + movsd(Address(rsp, 0), xmm0); + fld_d(Address(rsp, 0)); + jmp(L_2TAG_PACKET_6_0_2); + + bind(L_2TAG_PACKET_7_0_2); + movsd(xmm0, Address(rsp, 128)); + movsd(xmm1, Address(rsp, 136)); + mulsd(xmm0, xmm1); + movsd(Address(rsp, 0), xmm0); + fld_d(Address(rsp, 0)); + jmp(L_2TAG_PACKET_6_0_2); + + bind(L_2TAG_PACKET_0_0_2); + addl(eax, 16); + movl(edx, 32752); + andl(edx, eax); + cmpl(edx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_8_0_2); + testl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_9_0_2); + + bind(L_2TAG_PACKET_10_0_2); + movl(ecx, Address(rsp, 16)); + xorl(edx, edx); + testl(ecx, ecx); + movl(ecx, 1); + cmovl(Assembler::notEqual, edx, ecx); + orl(edx, Address(rsp, 20)); + cmpl(edx, 1072693248); + jcc(Assembler::equal, L_2TAG_PACKET_7_0_2); + movsd(xmm0, Address(rsp, 8)); + movsd(xmm3, Address(rsp, 8)); + movdl(edx, xmm3); + psrlq(xmm3, 32); + movdl(ecx, xmm3); + orl(edx, ecx); + cmpl(edx, 0); + jcc(Assembler::equal, L_2TAG_PACKET_11_0_2); + xorpd(xmm3, xmm3); + movl(eax, 18416); + pinsrw(xmm3, eax, 3); + mulsd(xmm0, xmm3); + xorpd(xmm2, xmm2); + movl(eax, 16368); + pinsrw(xmm2, eax, 3); + movdqu(xmm3, xmm0); + pextrw(eax, xmm0, 3); + por(xmm0, xmm2); + movl(ecx, 18416); + psllq(xmm0, 5); + movsd(xmm2, Address(tmp, 8256)); + psrlq(xmm0, 34); + rcpss(xmm0, xmm0); + psllq(xmm3, 12); + movdqu(xmm6, Address(tmp, 8240)); + psrlq(xmm3, 12); + mulss(xmm0, xmm7); + movl(edx, -1024); + movdl(xmm5, edx); + por(xmm3, xmm1); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + pand(xmm5, xmm3); + movl(rsi, 0); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + andl(eax, 32752); + subl(eax, 18416); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + jmp(L_2TAG_PACKET_4_0_2); + + bind(L_2TAG_PACKET_12_0_2); + movl(ecx, Address(rsp, 16)); + xorl(edx, edx); + testl(ecx, ecx); + movl(ecx, 1); + cmovl(Assembler::notEqual, edx, ecx); + orl(edx, Address(rsp, 20)); + cmpl(edx, 1072693248); + jcc(Assembler::equal, L_2TAG_PACKET_7_0_2); + movsd(xmm0, Address(rsp, 8)); + movsd(xmm3, Address(rsp, 8)); + movdl(edx, xmm3); + psrlq(xmm3, 32); + movdl(ecx, xmm3); + orl(edx, ecx); + cmpl(edx, 0); + jcc(Assembler::equal, L_2TAG_PACKET_11_0_2); + xorpd(xmm3, xmm3); + movl(eax, 18416); + pinsrw(xmm3, eax, 3); + mulsd(xmm0, xmm3); + xorpd(xmm2, xmm2); + movl(eax, 16368); + pinsrw(xmm2, eax, 3); + movdqu(xmm3, xmm0); + pextrw(eax, xmm0, 3); + por(xmm0, xmm2); + movl(ecx, 18416); + psllq(xmm0, 5); + movsd(xmm2, Address(tmp, 8256)); + psrlq(xmm0, 34); + rcpss(xmm0, xmm0); + psllq(xmm3, 12); + movdqu(xmm6, Address(tmp, 8240)); + psrlq(xmm3, 12); + mulss(xmm0, xmm7); + movl(edx, -1024); + movdl(xmm5, edx); + por(xmm3, xmm1); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + pand(xmm5, xmm3); + movl(rsi, INT_MIN); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + andl(eax, 32752); + subl(eax, 18416); + sarl(eax, 4); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + jmp(L_2TAG_PACKET_4_0_2); + + bind(L_2TAG_PACKET_5_0_2); + cmpl(eax, 0); + jcc(Assembler::less, L_2TAG_PACKET_13_0_2); + cmpl(eax, 752); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_14_0_2); + + bind(L_2TAG_PACKET_15_0_2); + addsd(xmm0, xmm7); + movsd(xmm2, Address(tmp, 12544)); + addpd(xmm3, xmm0); + xorpd(xmm6, xmm6); + movl(eax, 17080); + pinsrw(xmm6, eax, 3); + pshufd(xmm0, xmm3, 238); + addsd(xmm0, xmm3); + movdqu(xmm3, xmm5); + addsd(xmm5, xmm0); + movdqu(xmm4, xmm2); + subsd(xmm3, xmm5); + movdqu(xmm7, xmm5); + pand(xmm5, xmm2); + movdqu(xmm2, xmm1); + pand(xmm4, xmm1); + subsd(xmm7, xmm5); + addsd(xmm0, xmm3); + subsd(xmm1, xmm4); + mulsd(xmm4, xmm5); + addsd(xmm0, xmm7); + mulsd(xmm2, xmm0); + movdqu(xmm7, xmm6); + mulsd(xmm1, xmm5); + addsd(xmm6, xmm4); + movdl(eax, xmm6); + subsd(xmm6, xmm7); + addsd(xmm2, xmm1); + movdqu(xmm7, Address(tmp, 12480)); + movdqu(xmm3, Address(tmp, 12496)); + subsd(xmm4, xmm6); + pextrw(edx, xmm6, 3); + movl(ecx, eax); + andl(eax, 255); + addl(eax, eax); + movdqu(xmm5, Address(tmp, eax, Address::times_8, 8384)); + addsd(xmm2, xmm4); + sarl(ecx, 8); + movl(eax, ecx); + sarl(ecx, 1); + subl(eax, ecx); + shll(ecx, 20); + xorl(ecx, rsi); + movdl(xmm6, ecx); + movsd(xmm1, Address(tmp, 12512)); + andl(edx, 32767); + cmpl(edx, 16529); + jcc(Assembler::above, L_2TAG_PACKET_14_0_2); + pshufd(xmm0, xmm2, 68); + pshufd(xmm4, xmm2, 68); + mulpd(xmm0, xmm0); + mulpd(xmm7, xmm4); + pshufd(xmm6, xmm6, 17); + mulsd(xmm1, xmm2); + mulsd(xmm0, xmm0); + paddd(xmm5, xmm6); + addpd(xmm3, xmm7); + mulsd(xmm1, xmm5); + pshufd(xmm6, xmm5, 238); + mulpd(xmm0, xmm3); + addsd(xmm1, xmm6); + pshufd(xmm3, xmm0, 238); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + shll(eax, 4); + xorpd(xmm4, xmm4); + addl(eax, 16368); + pinsrw(xmm4, eax, 3); + addsd(xmm0, xmm1); + movl(rsi, Address(rsp, 24)); + addsd(xmm0, xmm3); + movdqu(xmm1, xmm0); + addsd(xmm0, xmm5); + mulsd(xmm0, xmm4); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_16_0_2); + cmpl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_18_0_2); + movsd(Address(rsp, 0), xmm0); + fld_d(Address(rsp, 0)); + jmp(L_2TAG_PACKET_6_0_2); + + bind(L_2TAG_PACKET_8_0_2); + movsd(xmm1, Address(rsp, 16)); + movsd(xmm0, Address(rsp, 8)); + movdqu(xmm2, xmm0); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_19_0_2); + addsd(xmm0, xmm0); + movdl(eax, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_20_0_2); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_20_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + movl(edx, 29); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_22_0_2); + movsd(xmm0, Address(rsp, 16)); + addpd(xmm0, xmm0); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_19_0_2); + movdl(eax, xmm1); + movdqu(xmm2, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_23_0_2); + pextrw(eax, xmm2, 3); + andl(eax, 32752); + cmpl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_24_0_2); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_22_0_2); + + bind(L_2TAG_PACKET_24_0_2); + pextrw(eax, xmm0, 3); + testl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_25_0_2); + testl(ecx, INT_MIN); + jcc(Assembler::notEqual, L_2TAG_PACKET_26_0_2); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_27_0_2); + movsd(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_28_0_2); + testl(eax, 2); + jcc(Assembler::notEqual, L_2TAG_PACKET_29_0_2); + jmp(L_2TAG_PACKET_28_0_2); + + bind(L_2TAG_PACKET_25_0_2); + shrl(ecx, 20); + andl(ecx, 2047); + cmpl(ecx, 1075); + jcc(Assembler::above, L_2TAG_PACKET_28_0_2); + jcc(Assembler::equal, L_2TAG_PACKET_30_0_2); + cmpl(ecx, 1074); + jcc(Assembler::above, L_2TAG_PACKET_27_0_2); + cmpl(ecx, 1023); + jcc(Assembler::below, L_2TAG_PACKET_28_0_2); + movsd(xmm1, Address(rsp, 16)); + movl(eax, 17208); + xorpd(xmm3, xmm3); + pinsrw(xmm3, eax, 3); + movdqu(xmm4, xmm3); + addsd(xmm3, xmm1); + subsd(xmm4, xmm3); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_28_0_2); + movdl(eax, xmm3); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_28_0_2); + + bind(L_2TAG_PACKET_29_0_2); + movsd(xmm1, Address(rsp, 16)); + pextrw(eax, xmm1, 3); + andl(eax, 32768); + jcc(Assembler::equal, L_2TAG_PACKET_18_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32768); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_28_0_2); + movsd(xmm1, Address(rsp, 16)); + pextrw(eax, xmm1, 3); + andl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_26_0_2); + + bind(L_2TAG_PACKET_31_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32752); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_30_0_2); + movsd(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_28_0_2); + jmp(L_2TAG_PACKET_29_0_2); + + bind(L_2TAG_PACKET_32_0_2); + movdl(eax, xmm1); + psrlq(xmm1, 20); + movdl(edx, xmm1); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_33_0_2); + movsd(xmm0, Address(rsp, 16)); + addsd(xmm0, xmm0); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_33_0_2); + movsd(xmm0, Address(rsp, 8)); + pextrw(eax, xmm0, 3); + cmpl(eax, 49136); + jcc(Assembler::notEqual, L_2TAG_PACKET_34_0_2); + movdl(ecx, xmm0); + psrlq(xmm0, 20); + movdl(edx, xmm0); + orl(ecx, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_34_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32760); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_34_0_2); + movsd(xmm1, Address(rsp, 16)); + andl(eax, 32752); + subl(eax, 16368); + pextrw(edx, xmm1, 3); + xorpd(xmm0, xmm0); + xorl(eax, edx); + andl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_18_0_2); + movl(ecx, 32752); + pinsrw(xmm0, ecx, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_35_0_2); + movdl(eax, xmm1); + cmpl(edx, 17184); + jcc(Assembler::above, L_2TAG_PACKET_36_0_2); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_37_0_2); + testl(eax, 2); + jcc(Assembler::equal, L_2TAG_PACKET_38_0_2); + jmp(L_2TAG_PACKET_39_0_2); + + bind(L_2TAG_PACKET_36_0_2); + testl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_38_0_2); + jmp(L_2TAG_PACKET_39_0_2); + + bind(L_2TAG_PACKET_9_0_2); + movsd(xmm2, Address(rsp, 8)); + movdl(eax, xmm2); + psrlq(xmm2, 31); + movdl(ecx, xmm2); + orl(eax, ecx); + jcc(Assembler::equal, L_2TAG_PACKET_11_0_2); + movsd(xmm1, Address(rsp, 16)); + pextrw(edx, xmm1, 3); + movdl(eax, xmm1); + movdqu(xmm2, xmm1); + psrlq(xmm2, 32); + movdl(ecx, xmm2); + addl(ecx, ecx); + orl(ecx, eax); + jcc(Assembler::equal, L_2TAG_PACKET_40_0_2); + andl(edx, 32752); + cmpl(edx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_32_0_2); + cmpl(edx, 17200); + jcc(Assembler::above, L_2TAG_PACKET_38_0_2); + cmpl(edx, 17184); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_35_0_2); + cmpl(edx, 16368); + jcc(Assembler::below, L_2TAG_PACKET_37_0_2); + movl(eax, 17208); + xorpd(xmm2, xmm2); + pinsrw(xmm2, eax, 3); + movdqu(xmm4, xmm2); + addsd(xmm2, xmm1); + subsd(xmm4, xmm2); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32767); + jcc(Assembler::notEqual, L_2TAG_PACKET_37_0_2); + movdl(eax, xmm2); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_38_0_2); + + bind(L_2TAG_PACKET_39_0_2); + xorpd(xmm1, xmm1); + movl(edx, 30704); + pinsrw(xmm1, edx, 3); + movsd(xmm2, Address(tmp, 8256)); + movsd(xmm4, Address(rsp, 8)); + pextrw(eax, xmm4, 3); + movl(edx, 8192); + movdl(xmm4, edx); + andl(eax, 32767); + subl(eax, 16); + jcc(Assembler::less, L_2TAG_PACKET_12_0_2); + movl(edx, eax); + andl(edx, 32752); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + addl(ecx, 16); + bsrl(ecx, ecx); + movl(rsi, INT_MIN); + jmp(L_2TAG_PACKET_1_0_2); + + bind(L_2TAG_PACKET_37_0_2); + xorpd(xmm1, xmm1); + movl(eax, 32752); + pinsrw(xmm1, eax, 3); + xorpd(xmm0, xmm0); + mulsd(xmm0, xmm1); + movl(edx, 28); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_38_0_2); + xorpd(xmm1, xmm1); + movl(edx, 30704); + pinsrw(xmm1, edx, 3); + movsd(xmm2, Address(tmp, 8256)); + movsd(xmm4, Address(rsp, 8)); + pextrw(eax, xmm4, 3); + movl(edx, 8192); + movdl(xmm4, edx); + andl(eax, 32767); + subl(eax, 16); + jcc(Assembler::less, L_2TAG_PACKET_10_0_2); + movl(edx, eax); + andl(edx, 32752); + subl(edx, 16368); + movl(ecx, edx); + sarl(edx, 31); + addl(ecx, edx); + xorl(ecx, edx); + addl(ecx, 16); + bsrl(ecx, ecx); + movl(rsi, 0); + jmp(L_2TAG_PACKET_1_0_2); + + bind(L_2TAG_PACKET_23_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_26_0_2); + xorpd(xmm0, xmm0); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_13_0_2); + addl(eax, 384); + cmpl(eax, 0); + jcc(Assembler::less, L_2TAG_PACKET_41_0_2); + mulsd(xmm5, xmm1); + addsd(xmm0, xmm7); + shrl(rsi, 31); + addpd(xmm3, xmm0); + pshufd(xmm0, xmm3, 238); + addsd(xmm3, xmm0); + movsd(xmm4, Address(tmp, rsi, Address::times_8, 12528)); + mulsd(xmm1, xmm3); + xorpd(xmm0, xmm0); + movl(eax, 16368); + shll(rsi, 15); + orl(eax, rsi); + pinsrw(xmm0, eax, 3); + addsd(xmm5, xmm1); + movl(rsi, Address(rsp, 24)); + mulsd(xmm5, xmm4); + addsd(xmm0, xmm5); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_41_0_2); + movl(rsi, Address(rsp, 24)); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_40_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_42_0_2); + xorpd(xmm0, xmm0); + movl(eax, 16368); + pinsrw(xmm0, eax, 3); + movl(edx, 26); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_11_0_2); + movsd(xmm1, Address(rsp, 16)); + movdqu(xmm2, xmm1); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + cmpl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_43_0_2); + movdl(eax, xmm2); + psrlq(xmm2, 20); + movdl(edx, xmm2); + orl(eax, edx); + jcc(Assembler::notEqual, L_2TAG_PACKET_22_0_2); + + bind(L_2TAG_PACKET_43_0_2); + movdl(eax, xmm1); + psrlq(xmm1, 32); + movdl(edx, xmm1); + movl(ecx, edx); + addl(edx, edx); + orl(eax, edx); + jcc(Assembler::equal, L_2TAG_PACKET_42_0_2); + shrl(edx, 21); + cmpl(edx, 1075); + jcc(Assembler::above, L_2TAG_PACKET_44_0_2); + jcc(Assembler::equal, L_2TAG_PACKET_45_0_2); + cmpl(edx, 1023); + jcc(Assembler::below, L_2TAG_PACKET_44_0_2); + movsd(xmm1, Address(rsp, 16)); + movl(eax, 17208); + xorpd(xmm3, xmm3); + pinsrw(xmm3, eax, 3); + movdqu(xmm4, xmm3); + addsd(xmm3, xmm1); + subsd(xmm4, xmm3); + addsd(xmm1, xmm4); + pextrw(eax, xmm1, 3); + andl(eax, 32752); + jcc(Assembler::notEqual, L_2TAG_PACKET_44_0_2); + movdl(eax, xmm3); + andl(eax, 1); + jcc(Assembler::equal, L_2TAG_PACKET_44_0_2); + + bind(L_2TAG_PACKET_46_0_2); + movsd(xmm0, Address(rsp, 8)); + testl(ecx, INT_MIN); + jcc(Assembler::notEqual, L_2TAG_PACKET_47_0_2); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_45_0_2); + movsd(xmm1, Address(rsp, 16)); + movdl(eax, xmm1); + testl(eax, 1); + jcc(Assembler::notEqual, L_2TAG_PACKET_46_0_2); + + bind(L_2TAG_PACKET_44_0_2); + testl(ecx, INT_MIN); + jcc(Assembler::equal, L_2TAG_PACKET_26_0_2); + xorpd(xmm0, xmm0); + + bind(L_2TAG_PACKET_47_0_2); + movl(eax, 16368); + xorpd(xmm1, xmm1); + pinsrw(xmm1, eax, 3); + divsd(xmm1, xmm0); + movdqu(xmm0, xmm1); + movl(edx, 27); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_14_0_2); + movsd(xmm2, Address(rsp, 8)); + movsd(xmm6, Address(rsp, 16)); + pextrw(eax, xmm2, 3); + pextrw(edx, xmm6, 3); + movl(ecx, 32752); + andl(ecx, edx); + cmpl(ecx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_48_0_2); + andl(eax, 32752); + subl(eax, 16368); + xorl(edx, eax); + testl(edx, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_49_0_2); + + bind(L_2TAG_PACKET_50_0_2); + movl(eax, 32736); + pinsrw(xmm0, eax, 3); + shrl(rsi, 16); + orl(eax, rsi); + pinsrw(xmm1, eax, 3); + movl(rsi, Address(rsp, 24)); + mulsd(xmm0, xmm1); + + bind(L_2TAG_PACKET_17_0_2); + movl(edx, 24); + + bind(L_2TAG_PACKET_21_0_2); + movsd(Address(rsp, 0), xmm0); + fld_d(Address(rsp, 0)); + jmp(L_2TAG_PACKET_6_0_2); + + bind(L_2TAG_PACKET_49_0_2); + movl(eax, 16); + pinsrw(xmm0, eax, 3); + mulsd(xmm0, xmm0); + testl(rsi, INT_MIN); + jcc(Assembler::equal, L_2TAG_PACKET_51_0_2); + movsd(xmm2, Address(tmp, 12560)); + xorpd(xmm0, xmm2); + + bind(L_2TAG_PACKET_51_0_2); + movl(rsi, Address(rsp, 24)); + movl(edx, 25); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_16_0_2); + pextrw(ecx, xmm5, 3); + pextrw(edx, xmm4, 3); + movl(eax, -1); + andl(ecx, 32752); + subl(ecx, 16368); + andl(edx, 32752); + addl(edx, ecx); + movl(ecx, -31); + sarl(edx, 4); + subl(ecx, edx); + jcc(Assembler::lessEqual, L_2TAG_PACKET_52_0_2); + cmpl(ecx, 20); + jcc(Assembler::above, L_2TAG_PACKET_53_0_2); + shll(eax); + + bind(L_2TAG_PACKET_52_0_2); + movdl(xmm0, eax); + psllq(xmm0, 32); + pand(xmm0, xmm5); + subsd(xmm5, xmm0); + addsd(xmm5, xmm1); + mulsd(xmm0, xmm4); + mulsd(xmm5, xmm4); + addsd(xmm0, xmm5); + + bind(L_2TAG_PACKET_53_0_2); + movl(edx, 25); + jmp(L_2TAG_PACKET_21_0_2); + + bind(L_2TAG_PACKET_2_0_2); + movzwl(ecx, Address(rsp, 22)); + movl(edx, INT_MIN); + movdl(xmm1, edx); + xorpd(xmm7, xmm7); + paddd(xmm0, xmm4); + psllq(xmm5, 32); + movdl(edx, xmm0); + psllq(xmm0, 29); + paddq(xmm1, xmm3); + pand(xmm5, xmm1); + andl(ecx, 32752); + cmpl(ecx, 16560); + jcc(Assembler::below, L_2TAG_PACKET_3_0_2); + pand(xmm0, xmm6); + subsd(xmm3, xmm5); + addl(eax, 16351); + shrl(eax, 4); + subl(eax, 1022); + cvtsi2sdl(xmm7, eax); + mulpd(xmm5, xmm0); + movsd(xmm4, Address(tmp, 0)); + mulsd(xmm3, xmm0); + movsd(xmm6, Address(tmp, 0)); + subsd(xmm5, xmm2); + movsd(xmm1, Address(tmp, 8)); + pshufd(xmm2, xmm3, 68); + unpcklpd(xmm5, xmm3); + addsd(xmm3, xmm5); + movsd(xmm0, Address(tmp, 8)); + andl(edx, 16760832); + shrl(edx, 10); + addpd(xmm7, Address(tmp, edx, Address::times_1, -3616)); + mulsd(xmm4, xmm5); + mulsd(xmm0, xmm5); + mulsd(xmm6, xmm2); + mulsd(xmm1, xmm2); + movdqu(xmm2, xmm5); + mulsd(xmm4, xmm5); + addsd(xmm5, xmm0); + movdqu(xmm0, xmm7); + addsd(xmm2, xmm3); + addsd(xmm7, xmm5); + mulsd(xmm6, xmm2); + subsd(xmm0, xmm7); + movdqu(xmm2, xmm7); + addsd(xmm7, xmm4); + addsd(xmm0, xmm5); + subsd(xmm2, xmm7); + addsd(xmm4, xmm2); + pshufd(xmm2, xmm5, 238); + movdqu(xmm5, xmm7); + addsd(xmm7, xmm2); + addsd(xmm4, xmm0); + movdqu(xmm0, Address(tmp, 8272)); + subsd(xmm5, xmm7); + addsd(xmm6, xmm4); + movdqu(xmm4, xmm7); + addsd(xmm5, xmm2); + addsd(xmm7, xmm1); + movdqu(xmm2, Address(tmp, 8336)); + subsd(xmm4, xmm7); + addsd(xmm6, xmm5); + addsd(xmm4, xmm1); + pshufd(xmm5, xmm7, 238); + movdqu(xmm1, xmm7); + addsd(xmm7, xmm5); + subsd(xmm1, xmm7); + addsd(xmm1, xmm5); + movdqu(xmm5, Address(tmp, 8352)); + pshufd(xmm3, xmm3, 68); + addsd(xmm6, xmm4); + addsd(xmm6, xmm1); + movdqu(xmm1, Address(tmp, 8304)); + mulpd(xmm0, xmm3); + mulpd(xmm2, xmm3); + pshufd(xmm4, xmm3, 68); + mulpd(xmm3, xmm3); + addpd(xmm0, xmm1); + addpd(xmm5, xmm2); + mulsd(xmm4, xmm3); + movsd(xmm2, Address(tmp, 16)); + mulpd(xmm3, xmm3); + movsd(xmm1, Address(rsp, 16)); + movzwl(ecx, Address(rsp, 22)); + mulpd(xmm0, xmm4); + pextrw(eax, xmm7, 3); + mulpd(xmm5, xmm4); + mulpd(xmm0, xmm3); + movsd(xmm4, Address(tmp, 8376)); + pand(xmm2, xmm7); + addsd(xmm5, xmm6); + subsd(xmm7, xmm2); + addpd(xmm5, xmm0); + andl(eax, 32752); + subl(eax, 16368); + andl(ecx, 32752); + cmpl(ecx, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_48_0_2); + addl(ecx, eax); + cmpl(ecx, 16576); + jcc(Assembler::aboveEqual, L_2TAG_PACKET_54_0_2); + pshufd(xmm0, xmm5, 238); + pand(xmm4, xmm1); + movdqu(xmm3, xmm1); + addsd(xmm5, xmm0); + subsd(xmm1, xmm4); + xorpd(xmm6, xmm6); + movl(edx, 17080); + pinsrw(xmm6, edx, 3); + addsd(xmm7, xmm5); + mulsd(xmm4, xmm2); + mulsd(xmm1, xmm2); + movdqu(xmm5, xmm6); + mulsd(xmm3, xmm7); + addsd(xmm6, xmm4); + addsd(xmm1, xmm3); + movdqu(xmm7, Address(tmp, 12480)); + movdl(edx, xmm6); + subsd(xmm6, xmm5); + movdqu(xmm3, Address(tmp, 12496)); + movsd(xmm2, Address(tmp, 12512)); + subsd(xmm4, xmm6); + movl(ecx, edx); + andl(edx, 255); + addl(edx, edx); + movdqu(xmm5, Address(tmp, edx, Address::times_8, 8384)); + addsd(xmm4, xmm1); + pextrw(edx, xmm6, 3); + shrl(ecx, 8); + movl(eax, ecx); + shrl(ecx, 1); + subl(eax, ecx); + shll(ecx, 20); + movdl(xmm6, ecx); + pshufd(xmm0, xmm4, 68); + pshufd(xmm1, xmm4, 68); + mulpd(xmm0, xmm0); + mulpd(xmm7, xmm1); + pshufd(xmm6, xmm6, 17); + mulsd(xmm2, xmm4); + andl(edx, 32767); + cmpl(edx, 16529); + jcc(Assembler::above, L_2TAG_PACKET_14_0_2); + mulsd(xmm0, xmm0); + paddd(xmm5, xmm6); + addpd(xmm3, xmm7); + mulsd(xmm2, xmm5); + pshufd(xmm6, xmm5, 238); + mulpd(xmm0, xmm3); + addsd(xmm2, xmm6); + pshufd(xmm3, xmm0, 238); + addl(eax, 1023); + shll(eax, 20); + orl(eax, rsi); + movdl(xmm4, eax); + mulsd(xmm0, xmm5); + mulsd(xmm3, xmm5); + addsd(xmm0, xmm2); + psllq(xmm4, 32); + addsd(xmm0, xmm3); + movdqu(xmm1, xmm0); + addsd(xmm0, xmm5); + movl(rsi, Address(rsp, 24)); + mulsd(xmm0, xmm4); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_16_0_2); + cmpl(eax, 32752); + jcc(Assembler::equal, L_2TAG_PACKET_17_0_2); + + bind(L_2TAG_PACKET_55_0_2); + movsd(Address(rsp, 0), xmm0); + fld_d(Address(rsp, 0)); + jmp(L_2TAG_PACKET_6_0_2); + + bind(L_2TAG_PACKET_48_0_2); + movl(rsi, Address(rsp, 24)); + + bind(L_2TAG_PACKET_56_0_2); + movsd(xmm0, Address(rsp, 8)); + movsd(xmm1, Address(rsp, 16)); + addsd(xmm1, xmm1); + xorpd(xmm2, xmm2); + movl(eax, 49136); + pinsrw(xmm2, eax, 3); + addsd(xmm2, xmm0); + pextrw(eax, xmm2, 3); + cmpl(eax, 0); + jcc(Assembler::notEqual, L_2TAG_PACKET_57_0_2); + xorpd(xmm0, xmm0); + movl(eax, 32760); + pinsrw(xmm0, eax, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_57_0_2); + movdl(edx, xmm1); + movdqu(xmm3, xmm1); + psrlq(xmm3, 20); + movdl(ecx, xmm3); + orl(ecx, edx); + jcc(Assembler::equal, L_2TAG_PACKET_58_0_2); + addsd(xmm1, xmm1); + movdqu(xmm0, xmm1); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_58_0_2); + pextrw(eax, xmm0, 3); + andl(eax, 32752); + pextrw(edx, xmm1, 3); + xorpd(xmm0, xmm0); + subl(eax, 16368); + xorl(eax, edx); + testl(eax, 32768); + jcc(Assembler::notEqual, L_2TAG_PACKET_18_0_2); + movl(edx, 32752); + pinsrw(xmm0, edx, 3); + jmp(L_2TAG_PACKET_18_0_2); + + bind(L_2TAG_PACKET_54_0_2); + pextrw(eax, xmm1, 3); + pextrw(ecx, xmm2, 3); + xorl(eax, ecx); + testl(eax, 32768); + jcc(Assembler::equal, L_2TAG_PACKET_50_0_2); + jmp(L_2TAG_PACKET_49_0_2); + + bind(L_2TAG_PACKET_6_0_2); + movl(tmp, Address(rsp, 64)); + +} + +#endif // !_LP64 diff --git a/hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp b/hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp index c2892ed2a88..e4cb8c6ce18 100644 --- a/hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp +++ b/hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp @@ -189,7 +189,7 @@ OopMap* RegisterSaver::save_live_registers(MacroAssembler* masm, int additional_ } // Save full ZMM registes(16..num_xmm_regs) base_addr = XSAVE_AREA_UPPERBANK; - int off = 0; + off = 0; int vector_len = Assembler::AVX_512bit; for (int n = 16; n < num_xmm_regs; n++) { __ evmovdqul(Address(rsp, base_addr+(off++*64)), as_XMMRegister(n), vector_len); @@ -199,7 +199,7 @@ OopMap* RegisterSaver::save_live_registers(MacroAssembler* masm, int additional_ if (VM_Version::supports_evex()) { // Save upper bank of ZMM registers(16..31) for double/float usage int base_addr = XSAVE_AREA_UPPERBANK; - int off = 0; + off = 0; for (int n = 16; n < num_xmm_regs; n++) { __ movsd(Address(rsp, base_addr+(off++*64)), as_XMMRegister(n)); } @@ -325,7 +325,7 @@ void RegisterSaver::restore_live_registers(MacroAssembler* masm, bool restore_ve assert(MaxVectorSize == 64, "only 512bit vectors are supported now"); } #else - assert(!save_vectors, "vectors are generated only by C2"); + assert(!restore_vectors, "vectors are generated only by C2"); #endif // On EVEX enabled targets everything is handled in pop fpu state diff --git a/hotspot/src/cpu/x86/vm/stubGenerator_x86_32.cpp b/hotspot/src/cpu/x86/vm/stubGenerator_x86_32.cpp index 41373cb3bfe..5ac5593f1c9 100644 --- a/hotspot/src/cpu/x86/vm/stubGenerator_x86_32.cpp +++ b/hotspot/src/cpu/x86/vm/stubGenerator_x86_32.cpp @@ -170,7 +170,7 @@ class StubGenerator: public StubCodeGenerator { // provide initial value for required masks if (UseAVX > 2) { __ movl(rbx, 0xffff); - __ kmovdl(k1, rbx); + __ kmovwl(k1, rbx); } // save and initialize %mxcsr @@ -798,7 +798,7 @@ class StubGenerator: public StubCodeGenerator { if (UseAVX > 2) { __ push(rbx); __ movl(rbx, 0xffff); - __ kmovdl(k1, rbx); + __ kmovwl(k1, rbx); __ pop(rbx); } // Copy 64-byte chunks @@ -2126,15 +2126,6 @@ class StubGenerator: public StubCodeGenerator { __ trigfunc('t'); __ ret(0); } - { - StubCodeMark mark(this, "StubRoutines", "pow"); - StubRoutines::_intrinsic_pow = (double (*)(double,double)) __ pc(); - - __ fld_d(Address(rsp, 12)); - __ fld_d(Address(rsp, 4)); - __ pow_with_fallback(0); - __ ret(0); - } } // AES intrinsic stubs @@ -2151,6 +2142,17 @@ class StubGenerator: public StubCodeGenerator { return start; } + address generate_counter_shuffle_mask() { + __ align(16); + StubCodeMark mark(this, "StubRoutines", "counter_shuffle_mask"); + address start = __ pc(); + __ emit_data(0x0c0d0e0f, relocInfo::none, 0); + __ emit_data(0x08090a0b, relocInfo::none, 0); + __ emit_data(0x04050607, relocInfo::none, 0); + __ emit_data(0x00010203, relocInfo::none, 0); + return start; + } + // Utility routine for loading a 128-bit key word in little endian format // can optionally specify that the shuffle mask is already in an xmmregister void load_key(XMMRegister xmmdst, Register key, int offset, XMMRegister xmm_shuf_mask=NULL) { @@ -2176,6 +2178,31 @@ class StubGenerator: public StubCodeGenerator { __ aesdec(xmmdst, xmmtmp); } + // Utility routine for increase 128bit counter (iv in CTR mode) + // XMM_128bit, D3, D2, D1, D0 + void inc_counter(Register reg, XMMRegister xmmdst, int inc_delta, Label& next_block) { + __ pextrd(reg, xmmdst, 0x0); + __ addl(reg, inc_delta); + __ pinsrd(xmmdst, reg, 0x0); + __ jcc(Assembler::carryClear, next_block); // jump if no carry + + __ pextrd(reg, xmmdst, 0x01); // Carry-> D1 + __ addl(reg, 0x01); + __ pinsrd(xmmdst, reg, 0x01); + __ jcc(Assembler::carryClear, next_block); // jump if no carry + + __ pextrd(reg, xmmdst, 0x02); // Carry-> D2 + __ addl(reg, 0x01); + __ pinsrd(xmmdst, reg, 0x02); + __ jcc(Assembler::carryClear, next_block); // jump if no carry + + __ pextrd(reg, xmmdst, 0x03); // Carry -> D3 + __ addl(reg, 0x01); + __ pinsrd(xmmdst, reg, 0x03); + + __ BIND(next_block); // next instruction + } + // Arguments: // @@ -2751,6 +2778,317 @@ class StubGenerator: public StubCodeGenerator { return start; } + + // CTR AES crypt. + // In 32-bit stub, parallelize 4 blocks at a time + // Arguments: + // + // Inputs: + // c_rarg0 - source byte array address + // c_rarg1 - destination byte array address + // c_rarg2 - K (key) in little endian int array + // c_rarg3 - counter vector byte array address + // c_rarg4 - input length + // + // Output: + // rax - input length + // + address generate_counterMode_AESCrypt_Parallel() { + assert(UseAES, "need AES instructions and misaligned SSE support"); + __ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", "counterMode_AESCrypt"); + address start = __ pc(); + const Register from = rsi; // source array address + const Register to = rdx; // destination array address + const Register key = rcx; // key array address + const Register counter = rdi; // counter byte array initialized from initvector array address + + // and left with the results of the last encryption block + const Register len_reg = rbx; + const Register pos = rax; + + __ enter(); // required for proper stackwalking of RuntimeStub frame + handleSOERegisters(true /*saving*/); // save rbx, rsi, rdi + + // For EVEX with VL and BW, provide a standard mask, VL = 128 will guide the merge + // context for the registers used, where all instructions below are using 128-bit mode + // On EVEX without VL and BW, these instructions will all be AVX. + if (VM_Version::supports_avx512vlbw()) { + __ movl(rdx, 0xffff); + __ kmovdl(k1, rdx); + } + + // load registers from incoming parameters + const Address from_param(rbp, 8+0); + const Address to_param (rbp, 8+4); + const Address key_param (rbp, 8+8); + const Address rvec_param (rbp, 8+12); + const Address len_param (rbp, 8+16); + const Address saved_counter_param(rbp, 8 + 20); + const Address used_addr_param(rbp, 8 + 24); + + __ movptr(from , from_param); + __ movptr(to , to_param); + //__ movptr(key, key_param); + //__ movptr(counter, rvec_param); + __ movptr(len_reg , len_param); + //__ movptr(pos, 0); + + // Use the partially used encrpyted counter from last invocation + Label L_exit_preLoop, L_preLoop_start; + + // Use the registers 'counter' and 'key' here in this preloop + // to hold of last 2 params 'used' and 'saved_encCounter_start' + Register used = counter; + Register saved_encCounter_start = key; + Register used_addr = saved_encCounter_start; + + __ movptr(used_addr, used_addr_param); + __ movptr(used, Address(used_addr, 0)); + __ movptr(saved_encCounter_start, saved_counter_param); + + __ BIND(L_preLoop_start); + __ cmpptr(used, 16); + __ jcc(Assembler::aboveEqual, L_exit_preLoop); + __ cmpptr(len_reg, 0); + __ jcc(Assembler::lessEqual, L_exit_preLoop); + __ movb(rax, Address(saved_encCounter_start, used)); + __ xorb(rax, Address(from, 0)); + __ movb(Address(to, 0), rax); + __ addptr(from, 1); + __ addptr(to, 1); + __ addptr(used, 1); + __ subptr(len_reg, 1); + + __ jmp(L_preLoop_start); + + __ BIND(L_exit_preLoop); + __ movptr(used_addr, used_addr_param); + __ movptr(used_addr, used_addr_param); + __ movl(Address(used_addr, 0), used); + + // load the parameters 'key' and 'counter' + __ movptr(key, key_param); + __ movptr(counter, rvec_param); + + // xmm register assignments for the loops below + const XMMRegister xmm_curr_counter = xmm0; + const XMMRegister xmm_counter_shuf_mask = xmm1; // need to be reloaded + const XMMRegister xmm_key_shuf_mask = xmm2; // need to be reloaded + const XMMRegister xmm_key = xmm3; + const XMMRegister xmm_result0 = xmm4; + const XMMRegister xmm_result1 = xmm5; + const XMMRegister xmm_result2 = xmm6; + const XMMRegister xmm_result3 = xmm7; + const XMMRegister xmm_from0 = xmm1; //reuse XMM register + const XMMRegister xmm_from1 = xmm2; + const XMMRegister xmm_from2 = xmm3; + const XMMRegister xmm_from3 = xmm4; + + //for key_128, key_192, key_256 + const int rounds[3] = {10, 12, 14}; + Label L_singleBlockLoopTop[3]; + Label L_multiBlock_loopTop[3]; + Label L_key192_top, L_key256_top; + Label L_incCounter[3][4]; // 3: different key length, 4: 4 blocks at a time + Label L_incCounter_single[3]; //for single block, key128, key192, key256 + Label L_processTail_insr[3], L_processTail_4_insr[3], L_processTail_2_insr[3], L_processTail_1_insr[3], L_processTail_exit_insr[3]; + Label L_processTail_extr[3], L_processTail_4_extr[3], L_processTail_2_extr[3], L_processTail_1_extr[3], L_processTail_exit_extr[3]; + + Label L_exit; + const int PARALLEL_FACTOR = 4; //because of the limited register number + + // initialize counter with initial counter + __ movdqu(xmm_curr_counter, Address(counter, 0x00)); + __ movdqu(xmm_counter_shuf_mask, ExternalAddress(StubRoutines::x86::counter_shuffle_mask_addr())); + __ pshufb(xmm_curr_counter, xmm_counter_shuf_mask); //counter is shuffled for increase + + // key length could be only {11, 13, 15} * 4 = {44, 52, 60} + __ movdqu(xmm_key_shuf_mask, ExternalAddress(StubRoutines::x86::key_shuffle_mask_addr())); + __ movl(rax, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT))); + __ cmpl(rax, 52); + __ jcc(Assembler::equal, L_key192_top); + __ cmpl(rax, 60); + __ jcc(Assembler::equal, L_key256_top); + + //key128 begins here + __ movptr(pos, 0); // init pos before L_multiBlock_loopTop + +#define CTR_DoFour(opc, src_reg) \ + __ opc(xmm_result0, src_reg); \ + __ opc(xmm_result1, src_reg); \ + __ opc(xmm_result2, src_reg); \ + __ opc(xmm_result3, src_reg); + + // k == 0 : generate code for key_128 + // k == 1 : generate code for key_192 + // k == 2 : generate code for key_256 + for (int k = 0; k < 3; ++k) { + //multi blocks starts here + __ align(OptoLoopAlignment); + __ BIND(L_multiBlock_loopTop[k]); + __ cmpptr(len_reg, PARALLEL_FACTOR * AESBlockSize); // see if at least PARALLEL_FACTOR blocks left + __ jcc(Assembler::less, L_singleBlockLoopTop[k]); + + __ movdqu(xmm_key_shuf_mask, ExternalAddress(StubRoutines::x86::key_shuffle_mask_addr())); + __ movdqu(xmm_counter_shuf_mask, ExternalAddress(StubRoutines::x86::counter_shuffle_mask_addr())); + + //load, then increase counters + CTR_DoFour(movdqa, xmm_curr_counter); + __ push(rbx); + inc_counter(rbx, xmm_result1, 0x01, L_incCounter[k][0]); + inc_counter(rbx, xmm_result2, 0x02, L_incCounter[k][1]); + inc_counter(rbx, xmm_result3, 0x03, L_incCounter[k][2]); + inc_counter(rbx, xmm_curr_counter, 0x04, L_incCounter[k][3]); + __ pop (rbx); + + load_key(xmm_key, key, 0x00, xmm_key_shuf_mask); // load Round 0 key. interleaving for better performance + + CTR_DoFour(pshufb, xmm_counter_shuf_mask); // after increased, shuffled counters back for PXOR + CTR_DoFour(pxor, xmm_key); //PXOR with Round 0 key + + for (int i = 1; i < rounds[k]; ++i) { + load_key(xmm_key, key, (0x10 * i), xmm_key_shuf_mask); + CTR_DoFour(aesenc, xmm_key); + } + load_key(xmm_key, key, (0x10 * rounds[k]), xmm_key_shuf_mask); + CTR_DoFour(aesenclast, xmm_key); + + // get next PARALLEL_FACTOR blocks into xmm_from registers + __ movdqu(xmm_from0, Address(from, pos, Address::times_1, 0 * AESBlockSize)); + __ movdqu(xmm_from1, Address(from, pos, Address::times_1, 1 * AESBlockSize)); + __ movdqu(xmm_from2, Address(from, pos, Address::times_1, 2 * AESBlockSize)); + + // PXOR with input text + __ pxor(xmm_result0, xmm_from0); //result0 is xmm4 + __ pxor(xmm_result1, xmm_from1); + __ pxor(xmm_result2, xmm_from2); + + // store PARALLEL_FACTOR results into the next 64 bytes of output + __ movdqu(Address(to, pos, Address::times_1, 0 * AESBlockSize), xmm_result0); + __ movdqu(Address(to, pos, Address::times_1, 1 * AESBlockSize), xmm_result1); + __ movdqu(Address(to, pos, Address::times_1, 2 * AESBlockSize), xmm_result2); + + // do it here after xmm_result0 is saved, because xmm_from3 reuse the same register of xmm_result0. + __ movdqu(xmm_from3, Address(from, pos, Address::times_1, 3 * AESBlockSize)); + __ pxor(xmm_result3, xmm_from3); + __ movdqu(Address(to, pos, Address::times_1, 3 * AESBlockSize), xmm_result3); + + __ addptr(pos, PARALLEL_FACTOR * AESBlockSize); // increase the length of crypt text + __ subptr(len_reg, PARALLEL_FACTOR * AESBlockSize); // decrease the remaining length + __ jmp(L_multiBlock_loopTop[k]); + + // singleBlock starts here + __ align(OptoLoopAlignment); + __ BIND(L_singleBlockLoopTop[k]); + __ cmpptr(len_reg, 0); + __ jcc(Assembler::equal, L_exit); + __ movdqu(xmm_key_shuf_mask, ExternalAddress(StubRoutines::x86::key_shuffle_mask_addr())); + __ movdqu(xmm_counter_shuf_mask, ExternalAddress(StubRoutines::x86::counter_shuffle_mask_addr())); + __ movdqa(xmm_result0, xmm_curr_counter); + load_key(xmm_key, key, 0x00, xmm_key_shuf_mask); + __ push(rbx);//rbx is used for increasing counter + inc_counter(rbx, xmm_curr_counter, 0x01, L_incCounter_single[k]); + __ pop (rbx); + __ pshufb(xmm_result0, xmm_counter_shuf_mask); + __ pxor(xmm_result0, xmm_key); + for (int i = 1; i < rounds[k]; i++) { + load_key(xmm_key, key, (0x10 * i), xmm_key_shuf_mask); + __ aesenc(xmm_result0, xmm_key); + } + load_key(xmm_key, key, (0x10 * rounds[k]), xmm_key_shuf_mask); + __ aesenclast(xmm_result0, xmm_key); + __ cmpptr(len_reg, AESBlockSize); + __ jcc(Assembler::less, L_processTail_insr[k]); + __ movdqu(xmm_from0, Address(from, pos, Address::times_1, 0 * AESBlockSize)); + __ pxor(xmm_result0, xmm_from0); + __ movdqu(Address(to, pos, Address::times_1, 0 * AESBlockSize), xmm_result0); + __ addptr(pos, AESBlockSize); + __ subptr(len_reg, AESBlockSize); + __ jmp(L_singleBlockLoopTop[k]); + + __ BIND(L_processTail_insr[k]); + __ addptr(pos, len_reg); + __ testptr(len_reg, 8); + __ jcc(Assembler::zero, L_processTail_4_insr[k]); + __ subptr(pos,8); + __ pinsrd(xmm_from0, Address(from, pos), 0); + __ pinsrd(xmm_from0, Address(from, pos, Address::times_1, 4), 1); + __ BIND(L_processTail_4_insr[k]); + __ testptr(len_reg, 4); + __ jcc(Assembler::zero, L_processTail_2_insr[k]); + __ subptr(pos,4); + __ pslldq(xmm_from0, 4); + __ pinsrd(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_2_insr[k]); + __ testptr(len_reg, 2); + __ jcc(Assembler::zero, L_processTail_1_insr[k]); + __ subptr(pos, 2); + __ pslldq(xmm_from0, 2); + __ pinsrw(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_1_insr[k]); + __ testptr(len_reg, 1); + __ jcc(Assembler::zero, L_processTail_exit_insr[k]); + __ subptr(pos, 1); + __ pslldq(xmm_from0, 1); + __ pinsrb(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_exit_insr[k]); + + __ movptr(saved_encCounter_start, saved_counter_param); + __ movdqu(Address(saved_encCounter_start, 0), xmm_result0); + __ pxor(xmm_result0, xmm_from0); + + __ testptr(len_reg, 8); + __ jcc(Assembler::zero, L_processTail_4_extr[k]); + __ pextrd(Address(to, pos), xmm_result0, 0); + __ pextrd(Address(to, pos, Address::times_1, 4), xmm_result0, 1); + __ psrldq(xmm_result0, 8); + __ addptr(pos, 8); + __ BIND(L_processTail_4_extr[k]); + __ testptr(len_reg, 4); + __ jcc(Assembler::zero, L_processTail_2_extr[k]); + __ pextrd(Address(to, pos), xmm_result0, 0); + __ psrldq(xmm_result0, 4); + __ addptr(pos, 4); + __ BIND(L_processTail_2_extr[k]); + __ testptr(len_reg, 2); + __ jcc(Assembler::zero, L_processTail_1_extr[k]); + __ pextrb(Address(to, pos), xmm_result0, 0); + __ pextrb(Address(to, pos, Address::times_1, 1), xmm_result0, 1); + __ psrldq(xmm_result0, 2); + __ addptr(pos, 2); + __ BIND(L_processTail_1_extr[k]); + __ testptr(len_reg, 1); + __ jcc(Assembler::zero, L_processTail_exit_extr[k]); + __ pextrb(Address(to, pos), xmm_result0, 0); + + __ BIND(L_processTail_exit_extr[k]); + __ movptr(used_addr, used_addr_param); + __ movl(Address(used_addr, 0), len_reg); + __ jmp(L_exit); + } + + __ BIND(L_exit); + __ movdqu(xmm_counter_shuf_mask, ExternalAddress(StubRoutines::x86::counter_shuffle_mask_addr())); + __ pshufb(xmm_curr_counter, xmm_counter_shuf_mask); //counter is shuffled back. + __ movdqu(Address(counter, 0), xmm_curr_counter); //save counter back + handleSOERegisters(false /*restoring*/); + __ movptr(rax, len_param); // return length + __ leave(); // required for proper stackwalking of RuntimeStub frame + __ ret(0); + + __ BIND (L_key192_top); + __ movptr(pos, 0); // init pos before L_multiBlock_loopTop + __ jmp(L_multiBlock_loopTop[1]); //key192 + + __ BIND (L_key256_top); + __ movptr(pos, 0); // init pos before L_multiBlock_loopTop + __ jmp(L_multiBlock_loopTop[2]); //key192 + + return start; + } + + // byte swap x86 long address generate_ghash_long_swap_mask() { __ align(CodeEntryAlignment); @@ -3082,6 +3420,30 @@ class StubGenerator: public StubCodeGenerator { } + address generate_libmPow() { + address start = __ pc(); + + const XMMRegister x0 = xmm0; + const XMMRegister x1 = xmm1; + const XMMRegister x2 = xmm2; + const XMMRegister x3 = xmm3; + + const XMMRegister x4 = xmm4; + const XMMRegister x5 = xmm5; + const XMMRegister x6 = xmm6; + const XMMRegister x7 = xmm7; + + const Register tmp = rbx; + + BLOCK_COMMENT("Entry:"); + __ enter(); // required for proper stackwalking of RuntimeStub frame + __ fast_pow(x0, x1, x2, x3, x4, x5, x6, x7, rax, rcx, rdx, tmp); + __ leave(); // required for proper stackwalking of RuntimeStub frame + __ ret(0); + + return start; + + } // Safefetch stubs. @@ -3310,6 +3672,7 @@ class StubGenerator: public StubCodeGenerator { if (VM_Version::supports_sse2()) { StubRoutines::_dexp = generate_libmExp(); StubRoutines::_dlog = generate_libmLog(); + StubRoutines::_dpow = generate_libmPow(); } } @@ -3344,6 +3707,11 @@ class StubGenerator: public StubCodeGenerator { StubRoutines::_cipherBlockChaining_decryptAESCrypt = generate_cipherBlockChaining_decryptAESCrypt(); } + if (UseAESCTRIntrinsics) { + StubRoutines::x86::_counter_shuffle_mask_addr = generate_counter_shuffle_mask(); + StubRoutines::_counterMode_AESCrypt = generate_counterMode_AESCrypt_Parallel(); + } + // Generate GHASH intrinsics code if (UseGHASHIntrinsics) { StubRoutines::x86::_ghash_long_swap_mask_addr = generate_ghash_long_swap_mask(); diff --git a/hotspot/src/cpu/x86/vm/stubGenerator_x86_64.cpp b/hotspot/src/cpu/x86/vm/stubGenerator_x86_64.cpp index fa9ae3a14e1..b3eb330ac9b 100644 --- a/hotspot/src/cpu/x86/vm/stubGenerator_x86_64.cpp +++ b/hotspot/src/cpu/x86/vm/stubGenerator_x86_64.cpp @@ -266,7 +266,7 @@ class StubGenerator: public StubCodeGenerator { __ movptr(r15_save, r15); if (UseAVX > 2) { __ movl(rbx, 0xffff); - __ kmovql(k1, rbx); + __ kmovwl(k1, rbx); } #ifdef _WIN64 int last_reg = 15; @@ -1350,7 +1350,7 @@ class StubGenerator: public StubCodeGenerator { Label L_end; if (UseAVX > 2) { __ movl(to, 0xffff); - __ kmovql(k1, to); + __ kmovwl(k1, to); } // Copy 64-bytes per iteration __ BIND(L_loop); @@ -1434,7 +1434,7 @@ class StubGenerator: public StubCodeGenerator { Label L_end; if (UseAVX > 2) { __ movl(to, 0xffff); - __ kmovql(k1, to); + __ kmovwl(k1, to); } // Copy 64-bytes per iteration __ BIND(L_loop); @@ -3025,21 +3025,6 @@ class StubGenerator: public StubCodeGenerator { __ addq(rsp, 8); __ ret(0); } - { - StubCodeMark mark(this, "StubRoutines", "pow"); - StubRoutines::_intrinsic_pow = (double (*)(double,double)) __ pc(); - - __ subq(rsp, 8); - __ movdbl(Address(rsp, 0), xmm1); - __ fld_d(Address(rsp, 0)); - __ movdbl(Address(rsp, 0), xmm0); - __ fld_d(Address(rsp, 0)); - __ pow_with_fallback(0); - __ fstp_d(Address(rsp, 0)); - __ movdbl(xmm0, Address(rsp, 0)); - __ addq(rsp, 8); - __ ret(0); - } } // AES intrinsic stubs @@ -3054,6 +3039,15 @@ class StubGenerator: public StubCodeGenerator { return start; } + address generate_counter_shuffle_mask() { + __ align(16); + StubCodeMark mark(this, "StubRoutines", "counter_shuffle_mask"); + address start = __ pc(); + __ emit_data64(0x08090a0b0c0d0e0f, relocInfo::none); + __ emit_data64(0x0001020304050607, relocInfo::none); + return start; + } + // Utility routine for loading a 128-bit key word in little endian format // can optionally specify that the shuffle mask is already in an xmmregister void load_key(XMMRegister xmmdst, Register key, int offset, XMMRegister xmm_shuf_mask=NULL) { @@ -3065,6 +3059,18 @@ class StubGenerator: public StubCodeGenerator { } } + // Utility routine for increase 128bit counter (iv in CTR mode) + void inc_counter(Register reg, XMMRegister xmmdst, int inc_delta, Label& next_block) { + __ pextrq(reg, xmmdst, 0x0); + __ addq(reg, inc_delta); + __ pinsrq(xmmdst, reg, 0x0); + __ jcc(Assembler::carryClear, next_block); // jump if no carry + __ pextrq(reg, xmmdst, 0x01); // Carry + __ addq(reg, 0x01); + __ pinsrq(xmmdst, reg, 0x01); //Carry end + __ BIND(next_block); // next instruction + } + // Arguments: // // Inputs: @@ -3715,6 +3721,328 @@ class StubGenerator: public StubCodeGenerator { return start; } + // This is a version of CTR/AES crypt which does 6 blocks in a loop at a time + // to hide instruction latency + // + // Arguments: + // + // Inputs: + // c_rarg0 - source byte array address + // c_rarg1 - destination byte array address + // c_rarg2 - K (key) in little endian int array + // c_rarg3 - counter vector byte array address + // Linux + // c_rarg4 - input length + // c_rarg5 - saved encryptedCounter start + // rbp + 6 * wordSize - saved used length + // Windows + // rbp + 6 * wordSize - input length + // rbp + 7 * wordSize - saved encryptedCounter start + // rbp + 8 * wordSize - saved used length + // + // Output: + // rax - input length + // + address generate_counterMode_AESCrypt_Parallel() { + assert(UseAES, "need AES instructions and misaligned SSE support"); + __ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", "counterMode_AESCrypt"); + address start = __ pc(); + const Register from = c_rarg0; // source array address + const Register to = c_rarg1; // destination array address + const Register key = c_rarg2; // key array address + const Register counter = c_rarg3; // counter byte array initialized from counter array address + // and left with the results of the last encryption block +#ifndef _WIN64 + const Register len_reg = c_rarg4; + const Register saved_encCounter_start = c_rarg5; + const Register used_addr = r10; + const Address used_mem(rbp, 2 * wordSize); + const Register used = r11; +#else + const Address len_mem(rbp, 6 * wordSize); // length is on stack on Win64 + const Address saved_encCounter_mem(rbp, 7 * wordSize); // length is on stack on Win64 + const Address used_mem(rbp, 8 * wordSize); // length is on stack on Win64 + const Register len_reg = r10; // pick the first volatile windows register + const Register saved_encCounter_start = r11; + const Register used_addr = r13; + const Register used = r14; +#endif + const Register pos = rax; + + const int PARALLEL_FACTOR = 6; + const XMMRegister xmm_counter_shuf_mask = xmm0; + const XMMRegister xmm_key_shuf_mask = xmm1; // used temporarily to swap key bytes up front + const XMMRegister xmm_curr_counter = xmm2; + + const XMMRegister xmm_key_tmp0 = xmm3; + const XMMRegister xmm_key_tmp1 = xmm4; + + // registers holding the four results in the parallelized loop + const XMMRegister xmm_result0 = xmm5; + const XMMRegister xmm_result1 = xmm6; + const XMMRegister xmm_result2 = xmm7; + const XMMRegister xmm_result3 = xmm8; + const XMMRegister xmm_result4 = xmm9; + const XMMRegister xmm_result5 = xmm10; + + const XMMRegister xmm_from0 = xmm11; + const XMMRegister xmm_from1 = xmm12; + const XMMRegister xmm_from2 = xmm13; + const XMMRegister xmm_from3 = xmm14; //the last one is xmm14. we have to preserve it on WIN64. + const XMMRegister xmm_from4 = xmm3; //reuse xmm3~4. Because xmm_key_tmp0~1 are useless when loading input text + const XMMRegister xmm_from5 = xmm4; + + //for key_128, key_192, key_256 + const int rounds[3] = {10, 12, 14}; + Label L_exit_preLoop, L_preLoop_start; + Label L_multiBlock_loopTop[3]; + Label L_singleBlockLoopTop[3]; + Label L__incCounter[3][6]; //for 6 blocks + Label L__incCounter_single[3]; //for single block, key128, key192, key256 + Label L_processTail_insr[3], L_processTail_4_insr[3], L_processTail_2_insr[3], L_processTail_1_insr[3], L_processTail_exit_insr[3]; + Label L_processTail_extr[3], L_processTail_4_extr[3], L_processTail_2_extr[3], L_processTail_1_extr[3], L_processTail_exit_extr[3]; + + Label L_exit; + + __ enter(); // required for proper stackwalking of RuntimeStub frame + + // For EVEX with VL and BW, provide a standard mask, VL = 128 will guide the merge + // context for the registers used, where all instructions below are using 128-bit mode + // On EVEX without VL and BW, these instructions will all be AVX. + if (VM_Version::supports_avx512vlbw()) { + __ movl(rax, 0xffff); + __ kmovql(k1, rax); + } + +#ifdef _WIN64 + // save the xmm registers which must be preserved 6-14 + const int XMM_REG_NUM_KEY_LAST = 14; + __ subptr(rsp, -rsp_after_call_off * wordSize); + for (int i = 6; i <= XMM_REG_NUM_KEY_LAST; i++) { + __ movdqu(xmm_save(i), as_XMMRegister(i)); + } + + const Address r13_save(rbp, rdi_off * wordSize); + const Address r14_save(rbp, rsi_off * wordSize); + + __ movptr(r13_save, r13); + __ movptr(r14_save, r14); + + // on win64, fill len_reg from stack position + __ movl(len_reg, len_mem); + __ movptr(saved_encCounter_start, saved_encCounter_mem); + __ movptr(used_addr, used_mem); + __ movl(used, Address(used_addr, 0)); +#else + __ push(len_reg); // Save + __ movptr(used_addr, used_mem); + __ movl(used, Address(used_addr, 0)); +#endif + + __ push(rbx); // Save RBX + __ movdqu(xmm_curr_counter, Address(counter, 0x00)); // initialize counter with initial counter + __ movdqu(xmm_counter_shuf_mask, ExternalAddress(StubRoutines::x86::counter_shuffle_mask_addr())); + __ pshufb(xmm_curr_counter, xmm_counter_shuf_mask); //counter is shuffled + __ movptr(pos, 0); + + // Use the partially used encrpyted counter from last invocation + __ BIND(L_preLoop_start); + __ cmpptr(used, 16); + __ jcc(Assembler::aboveEqual, L_exit_preLoop); + __ cmpptr(len_reg, 0); + __ jcc(Assembler::lessEqual, L_exit_preLoop); + __ movb(rbx, Address(saved_encCounter_start, used)); + __ xorb(rbx, Address(from, pos)); + __ movb(Address(to, pos), rbx); + __ addptr(pos, 1); + __ addptr(used, 1); + __ subptr(len_reg, 1); + + __ jmp(L_preLoop_start); + + __ BIND(L_exit_preLoop); + __ movl(Address(used_addr, 0), used); + + // key length could be only {11, 13, 15} * 4 = {44, 52, 60} + __ movdqu(xmm_key_shuf_mask, ExternalAddress(StubRoutines::x86::key_shuffle_mask_addr())); + __ movl(rbx, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT))); + __ cmpl(rbx, 52); + __ jcc(Assembler::equal, L_multiBlock_loopTop[1]); + __ cmpl(rbx, 60); + __ jcc(Assembler::equal, L_multiBlock_loopTop[2]); + +#define CTR_DoSix(opc, src_reg) \ + __ opc(xmm_result0, src_reg); \ + __ opc(xmm_result1, src_reg); \ + __ opc(xmm_result2, src_reg); \ + __ opc(xmm_result3, src_reg); \ + __ opc(xmm_result4, src_reg); \ + __ opc(xmm_result5, src_reg); + + // k == 0 : generate code for key_128 + // k == 1 : generate code for key_192 + // k == 2 : generate code for key_256 + for (int k = 0; k < 3; ++k) { + //multi blocks starts here + __ align(OptoLoopAlignment); + __ BIND(L_multiBlock_loopTop[k]); + __ cmpptr(len_reg, PARALLEL_FACTOR * AESBlockSize); // see if at least PARALLEL_FACTOR blocks left + __ jcc(Assembler::less, L_singleBlockLoopTop[k]); + load_key(xmm_key_tmp0, key, 0x00, xmm_key_shuf_mask); + + //load, then increase counters + CTR_DoSix(movdqa, xmm_curr_counter); + inc_counter(rbx, xmm_result1, 0x01, L__incCounter[k][0]); + inc_counter(rbx, xmm_result2, 0x02, L__incCounter[k][1]); + inc_counter(rbx, xmm_result3, 0x03, L__incCounter[k][2]); + inc_counter(rbx, xmm_result4, 0x04, L__incCounter[k][3]); + inc_counter(rbx, xmm_result5, 0x05, L__incCounter[k][4]); + inc_counter(rbx, xmm_curr_counter, 0x06, L__incCounter[k][5]); + CTR_DoSix(pshufb, xmm_counter_shuf_mask); // after increased, shuffled counters back for PXOR + CTR_DoSix(pxor, xmm_key_tmp0); //PXOR with Round 0 key + + //load two ROUND_KEYs at a time + for (int i = 1; i < rounds[k]; ) { + load_key(xmm_key_tmp1, key, (0x10 * i), xmm_key_shuf_mask); + load_key(xmm_key_tmp0, key, (0x10 * (i+1)), xmm_key_shuf_mask); + CTR_DoSix(aesenc, xmm_key_tmp1); + i++; + if (i != rounds[k]) { + CTR_DoSix(aesenc, xmm_key_tmp0); + } else { + CTR_DoSix(aesenclast, xmm_key_tmp0); + } + i++; + } + + // get next PARALLEL_FACTOR blocks into xmm_result registers + __ movdqu(xmm_from0, Address(from, pos, Address::times_1, 0 * AESBlockSize)); + __ movdqu(xmm_from1, Address(from, pos, Address::times_1, 1 * AESBlockSize)); + __ movdqu(xmm_from2, Address(from, pos, Address::times_1, 2 * AESBlockSize)); + __ movdqu(xmm_from3, Address(from, pos, Address::times_1, 3 * AESBlockSize)); + __ movdqu(xmm_from4, Address(from, pos, Address::times_1, 4 * AESBlockSize)); + __ movdqu(xmm_from5, Address(from, pos, Address::times_1, 5 * AESBlockSize)); + + __ pxor(xmm_result0, xmm_from0); + __ pxor(xmm_result1, xmm_from1); + __ pxor(xmm_result2, xmm_from2); + __ pxor(xmm_result3, xmm_from3); + __ pxor(xmm_result4, xmm_from4); + __ pxor(xmm_result5, xmm_from5); + + // store 6 results into the next 64 bytes of output + __ movdqu(Address(to, pos, Address::times_1, 0 * AESBlockSize), xmm_result0); + __ movdqu(Address(to, pos, Address::times_1, 1 * AESBlockSize), xmm_result1); + __ movdqu(Address(to, pos, Address::times_1, 2 * AESBlockSize), xmm_result2); + __ movdqu(Address(to, pos, Address::times_1, 3 * AESBlockSize), xmm_result3); + __ movdqu(Address(to, pos, Address::times_1, 4 * AESBlockSize), xmm_result4); + __ movdqu(Address(to, pos, Address::times_1, 5 * AESBlockSize), xmm_result5); + + __ addptr(pos, PARALLEL_FACTOR * AESBlockSize); // increase the length of crypt text + __ subptr(len_reg, PARALLEL_FACTOR * AESBlockSize); // decrease the remaining length + __ jmp(L_multiBlock_loopTop[k]); + + // singleBlock starts here + __ align(OptoLoopAlignment); + __ BIND(L_singleBlockLoopTop[k]); + __ cmpptr(len_reg, 0); + __ jcc(Assembler::lessEqual, L_exit); + load_key(xmm_key_tmp0, key, 0x00, xmm_key_shuf_mask); + __ movdqa(xmm_result0, xmm_curr_counter); + inc_counter(rbx, xmm_curr_counter, 0x01, L__incCounter_single[k]); + __ pshufb(xmm_result0, xmm_counter_shuf_mask); + __ pxor(xmm_result0, xmm_key_tmp0); + for (int i = 1; i < rounds[k]; i++) { + load_key(xmm_key_tmp0, key, (0x10 * i), xmm_key_shuf_mask); + __ aesenc(xmm_result0, xmm_key_tmp0); + } + load_key(xmm_key_tmp0, key, (rounds[k] * 0x10), xmm_key_shuf_mask); + __ aesenclast(xmm_result0, xmm_key_tmp0); + __ cmpptr(len_reg, AESBlockSize); + __ jcc(Assembler::less, L_processTail_insr[k]); + __ movdqu(xmm_from0, Address(from, pos, Address::times_1, 0 * AESBlockSize)); + __ pxor(xmm_result0, xmm_from0); + __ movdqu(Address(to, pos, Address::times_1, 0 * AESBlockSize), xmm_result0); + __ addptr(pos, AESBlockSize); + __ subptr(len_reg, AESBlockSize); + __ jmp(L_singleBlockLoopTop[k]); + __ BIND(L_processTail_insr[k]); + __ addptr(pos, len_reg); + __ testptr(len_reg, 8); + __ jcc(Assembler::zero, L_processTail_4_insr[k]); + __ subptr(pos,8); + __ pinsrq(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_4_insr[k]); + __ testptr(len_reg, 4); + __ jcc(Assembler::zero, L_processTail_2_insr[k]); + __ subptr(pos,4); + __ pslldq(xmm_from0, 4); + __ pinsrd(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_2_insr[k]); + __ testptr(len_reg, 2); + __ jcc(Assembler::zero, L_processTail_1_insr[k]); + __ subptr(pos, 2); + __ pslldq(xmm_from0, 2); + __ pinsrw(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_1_insr[k]); + __ testptr(len_reg, 1); + __ jcc(Assembler::zero, L_processTail_exit_insr[k]); + __ subptr(pos, 1); + __ pslldq(xmm_from0, 1); + __ pinsrb(xmm_from0, Address(from, pos), 0); + __ BIND(L_processTail_exit_insr[k]); + + __ movdqu(Address(saved_encCounter_start, 0), xmm_result0); + __ pxor(xmm_result0, xmm_from0); + + __ testptr(len_reg, 8); + __ jcc(Assembler::zero, L_processTail_4_extr[k]); + __ pextrq(Address(to, pos), xmm_result0, 0); + __ psrldq(xmm_result0, 8); + __ addptr(pos, 8); + __ BIND(L_processTail_4_extr[k]); + __ testptr(len_reg, 4); + __ jcc(Assembler::zero, L_processTail_2_extr[k]); + __ pextrd(Address(to, pos), xmm_result0, 0); + __ psrldq(xmm_result0, 4); + __ addptr(pos, 4); + __ BIND(L_processTail_2_extr[k]); + __ testptr(len_reg, 2); + __ jcc(Assembler::zero, L_processTail_1_extr[k]); + __ pextrw(Address(to, pos), xmm_result0, 0); + __ psrldq(xmm_result0, 2); + __ addptr(pos, 2); + __ BIND(L_processTail_1_extr[k]); + __ testptr(len_reg, 1); + __ jcc(Assembler::zero, L_processTail_exit_extr[k]); + __ pextrb(Address(to, pos), xmm_result0, 0); + + __ BIND(L_processTail_exit_extr[k]); + __ movl(Address(used_addr, 0), len_reg); + __ jmp(L_exit); + + } + + __ BIND(L_exit); + __ pshufb(xmm_curr_counter, xmm_counter_shuf_mask); //counter is shuffled back. + __ movdqu(Address(counter, 0), xmm_curr_counter); //save counter back + __ pop(rbx); // pop the saved RBX. +#ifdef _WIN64 + // restore regs belonging to calling function + for (int i = 6; i <= XMM_REG_NUM_KEY_LAST; i++) { + __ movdqu(as_XMMRegister(i), xmm_save(i)); + } + __ movl(rax, len_mem); + __ movptr(r13, r13_save); + __ movptr(r14, r14_save); +#else + __ pop(rax); // return 'len' +#endif + __ leave(); // required for proper stackwalking of RuntimeStub frame + __ ret(0); + return start; + } // byte swap x86 long address generate_ghash_long_swap_mask() { @@ -4054,6 +4382,54 @@ class StubGenerator: public StubCodeGenerator { return start; } + /** + * Arguments: + * + * Input: + * c_rarg0 - obja address + * c_rarg1 - objb address + * c_rarg3 - length length + * c_rarg4 - scale log2_array_indxscale + */ + address generate_vectorizedMismatch() { + __ align(CodeEntryAlignment); + StubCodeMark mark(this, "StubRoutines", "vectorizedMismatch"); + address start = __ pc(); + + BLOCK_COMMENT("Entry:"); + __ enter(); + +#ifdef _WIN64 // Win64: rcx, rdx, r8, r9 (c_rarg0, c_rarg1, ...) + const Register scale = c_rarg0; //rcx, will exchange with r9 + const Register objb = c_rarg1; //rdx + const Register length = c_rarg2; //r8 + const Register obja = c_rarg3; //r9 + __ xchgq(obja, scale); //now obja and scale contains the correct contents + + const Register tmp1 = r10; + const Register tmp2 = r11; +#endif +#ifndef _WIN64 // Unix: rdi, rsi, rdx, rcx, r8, r9 (c_rarg0, c_rarg1, ...) + const Register obja = c_rarg0; //U:rdi + const Register objb = c_rarg1; //U:rsi + const Register length = c_rarg2; //U:rdx + const Register scale = c_rarg3; //U:rcx + const Register tmp1 = r8; + const Register tmp2 = r9; +#endif + const Register result = rax; //return value + const XMMRegister vec0 = xmm0; + const XMMRegister vec1 = xmm1; + const XMMRegister vec2 = xmm2; + + __ vectorized_mismatch(obja, objb, length, scale, result, tmp1, tmp2, vec0, vec1, vec2); + + __ leave(); + __ ret(0); + + return start; + } + /** * Arguments: * @@ -4235,6 +4611,48 @@ class StubGenerator: public StubCodeGenerator { } + address generate_libmPow() { + address start = __ pc(); + + const XMMRegister x0 = xmm0; + const XMMRegister x1 = xmm1; + const XMMRegister x2 = xmm2; + const XMMRegister x3 = xmm3; + + const XMMRegister x4 = xmm4; + const XMMRegister x5 = xmm5; + const XMMRegister x6 = xmm6; + const XMMRegister x7 = xmm7; + + const Register tmp1 = r8; + const Register tmp2 = r9; + const Register tmp3 = r10; + const Register tmp4 = r11; + + BLOCK_COMMENT("Entry:"); + __ enter(); // required for proper stackwalking of RuntimeStub frame + +#ifdef _WIN64 + // save the xmm registers which must be preserved 6-7 + __ subptr(rsp, 4 * wordSize); + __ movdqu(Address(rsp, 0), xmm6); + __ movdqu(Address(rsp, 2 * wordSize), xmm7); +#endif + __ fast_pow(x0, x1, x2, x3, x4, x5, x6, x7, rax, rcx, rdx, tmp1, tmp2, tmp3, tmp4); + +#ifdef _WIN64 + // restore xmm regs belonging to calling function + __ movdqu(xmm6, Address(rsp, 0)); + __ movdqu(xmm7, Address(rsp, 2 * wordSize)); + __ addptr(rsp, 4 * wordSize); +#endif + + __ leave(); // required for proper stackwalking of RuntimeStub frame + __ ret(0); + + return start; + + } #undef __ #define __ masm-> @@ -4430,6 +4848,7 @@ class StubGenerator: public StubCodeGenerator { if (VM_Version::supports_sse2()) { StubRoutines::_dexp = generate_libmExp(); StubRoutines::_dlog = generate_libmLog(); + StubRoutines::_dpow = generate_libmPow(); } } @@ -4479,12 +4898,15 @@ class StubGenerator: public StubCodeGenerator { // don't bother generating these AES intrinsic stubs unless global flag is set if (UseAESIntrinsics) { StubRoutines::x86::_key_shuffle_mask_addr = generate_key_shuffle_mask(); // needed by the others - StubRoutines::_aescrypt_encryptBlock = generate_aescrypt_encryptBlock(); StubRoutines::_aescrypt_decryptBlock = generate_aescrypt_decryptBlock(); StubRoutines::_cipherBlockChaining_encryptAESCrypt = generate_cipherBlockChaining_encryptAESCrypt(); StubRoutines::_cipherBlockChaining_decryptAESCrypt = generate_cipherBlockChaining_decryptAESCrypt_Parallel(); } + if (UseAESCTRIntrinsics){ + StubRoutines::x86::_counter_shuffle_mask_addr = generate_counter_shuffle_mask(); + StubRoutines::_counterMode_AESCrypt = generate_counterMode_AESCrypt_Parallel(); + } // Generate GHASH intrinsics code if (UseGHASHIntrinsics) { @@ -4510,7 +4932,9 @@ class StubGenerator: public StubCodeGenerator { if (UseMulAddIntrinsic) { StubRoutines::_mulAdd = generate_mulAdd(); } - + if (UseVectorizedMismatchIntrinsic) { + StubRoutines::_vectorizedMismatch = generate_vectorizedMismatch(); + } #ifndef _WINDOWS if (UseMontgomeryMultiplyIntrinsic) { StubRoutines::_montgomeryMultiply diff --git a/hotspot/src/cpu/x86/vm/stubRoutines_x86.cpp b/hotspot/src/cpu/x86/vm/stubRoutines_x86.cpp index d7d2ced2cf0..3bc199b1db5 100644 --- a/hotspot/src/cpu/x86/vm/stubRoutines_x86.cpp +++ b/hotspot/src/cpu/x86/vm/stubRoutines_x86.cpp @@ -34,6 +34,7 @@ address StubRoutines::x86::_verify_mxcsr_entry = NULL; address StubRoutines::x86::_key_shuffle_mask_addr = NULL; +address StubRoutines::x86::_counter_shuffle_mask_addr = NULL; address StubRoutines::x86::_ghash_long_swap_mask_addr = NULL; address StubRoutines::x86::_ghash_byte_swap_mask_addr = NULL; diff --git a/hotspot/src/cpu/x86/vm/stubRoutines_x86.hpp b/hotspot/src/cpu/x86/vm/stubRoutines_x86.hpp index 7e236967fb8..5b2bf4bdf91 100644 --- a/hotspot/src/cpu/x86/vm/stubRoutines_x86.hpp +++ b/hotspot/src/cpu/x86/vm/stubRoutines_x86.hpp @@ -33,6 +33,10 @@ static address _verify_mxcsr_entry; // shuffle mask for fixing up 128-bit words consisting of big-endian 32-bit integers static address _key_shuffle_mask_addr; + + //shuffle mask for big-endian 128-bit integers + static address _counter_shuffle_mask_addr; + // masks and table for CRC32 static uint64_t _crc_by128_masks[]; static juint _crc_table[]; @@ -45,9 +49,9 @@ public: static address verify_mxcsr_entry() { return _verify_mxcsr_entry; } static address key_shuffle_mask_addr() { return _key_shuffle_mask_addr; } + static address counter_shuffle_mask_addr() { return _counter_shuffle_mask_addr; } static address crc_by128_masks_addr() { return (address)_crc_by128_masks; } static address ghash_long_swap_mask_addr() { return _ghash_long_swap_mask_addr; } static address ghash_byte_swap_mask_addr() { return _ghash_byte_swap_mask_addr; } static void generate_CRC32C_table(bool is_pclmulqdq_supported); - #endif // CPU_X86_VM_STUBROUTINES_X86_32_HPP diff --git a/hotspot/src/cpu/x86/vm/stubRoutines_x86_32.hpp b/hotspot/src/cpu/x86/vm/stubRoutines_x86_32.hpp index 1599939ebcf..c1ea0223389 100644 --- a/hotspot/src/cpu/x86/vm/stubRoutines_x86_32.hpp +++ b/hotspot/src/cpu/x86/vm/stubRoutines_x86_32.hpp @@ -31,7 +31,7 @@ enum platform_dependent_constants { code_size1 = 9000, // simply increase if too small (assembler will crash if too small) - code_size2 = 30000 // simply increase if too small (assembler will crash if too small) + code_size2 = 33800 // simply increase if too small (assembler will crash if too small) }; class x86 { diff --git a/hotspot/src/cpu/x86/vm/stubRoutines_x86_64.hpp b/hotspot/src/cpu/x86/vm/stubRoutines_x86_64.hpp index d8c50c82b23..6f39276dc05 100644 --- a/hotspot/src/cpu/x86/vm/stubRoutines_x86_64.hpp +++ b/hotspot/src/cpu/x86/vm/stubRoutines_x86_64.hpp @@ -33,7 +33,7 @@ static bool returns_to_call_stub(address return_pc) { return return_pc == _ enum platform_dependent_constants { code_size1 = 19000, // simply increase if too small (assembler will crash if too small) - code_size2 = 32000 // simply increase if too small (assembler will crash if too small) + code_size2 = 35000 // simply increase if too small (assembler will crash if too small) }; class x86 { diff --git a/hotspot/src/cpu/x86/vm/vmStructs_x86.hpp b/hotspot/src/cpu/x86/vm/vmStructs_x86.hpp index a337ab14590..40e6e431882 100644 --- a/hotspot/src/cpu/x86/vm/vmStructs_x86.hpp +++ b/hotspot/src/cpu/x86/vm/vmStructs_x86.hpp @@ -1,5 +1,5 @@ /* - * Copyright (c) 2001, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2001, 2015, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -30,20 +30,9 @@ // referenced by vmStructs.cpp. #define VM_STRUCTS_CPU(nonstatic_field, static_field, unchecked_nonstatic_field, volatile_nonstatic_field, nonproduct_nonstatic_field, c2_nonstatic_field, unchecked_c1_static_field, unchecked_c2_static_field) \ - \ - /******************************/ \ - /* JavaCallWrapper */ \ - /******************************/ \ - /******************************/ \ - /* JavaFrameAnchor */ \ - /******************************/ \ - volatile_nonstatic_field(JavaFrameAnchor, _last_Java_fp, intptr_t*) \ - static_field(VM_Version, _cpuFeatures, uint64_t) - - + volatile_nonstatic_field(JavaFrameAnchor, _last_Java_fp, intptr_t*) #define VM_TYPES_CPU(declare_type, declare_toplevel_type, declare_oop_type, declare_integer_type, declare_unsigned_integer_type, declare_c1_toplevel_type, declare_c2_type, declare_c2_toplevel_type) \ - declare_toplevel_type(VM_Version) #define VM_INT_CONSTANTS_CPU(declare_constant, declare_preprocessor_constant, declare_c1_constant, declare_c2_constant, declare_c2_preprocessor_constant) \ LP64_ONLY(declare_constant(frame::arg_reg_save_area_bytes)) \ diff --git a/hotspot/src/cpu/x86/vm/vm_version_x86.cpp b/hotspot/src/cpu/x86/vm/vm_version_x86.cpp index eed88e81342..d31b0e5acd2 100644 --- a/hotspot/src/cpu/x86/vm/vm_version_x86.cpp +++ b/hotspot/src/cpu/x86/vm/vm_version_x86.cpp @@ -35,9 +35,7 @@ int VM_Version::_cpu; int VM_Version::_model; int VM_Version::_stepping; -uint64_t VM_Version::_cpuFeatures; -const char* VM_Version::_features_str = ""; -VM_Version::CpuidInfo VM_Version::_cpuid_info = { 0, }; +VM_Version::CpuidInfo VM_Version::_cpuid_info = { 0, }; // Address of instruction which causes SEGV address VM_Version::_cpuinfo_segv_addr = 0; @@ -468,28 +466,26 @@ void VM_Version::get_processor_features() { _cpu = 4; // 486 by default _model = 0; _stepping = 0; - _cpuFeatures = 0; + _features = 0; _logical_processors_per_package = 1; // i486 internal cache is both I&D and has a 16-byte line size _L1_data_cache_line_size = 16; - if (!Use486InstrsOnly) { - // Get raw processor info + // Get raw processor info - get_cpu_info_stub(&_cpuid_info); + get_cpu_info_stub(&_cpuid_info); - assert_is_initialized(); - _cpu = extended_cpu_family(); - _model = extended_cpu_model(); - _stepping = cpu_stepping(); + assert_is_initialized(); + _cpu = extended_cpu_family(); + _model = extended_cpu_model(); + _stepping = cpu_stepping(); - if (cpu_family() > 4) { // it supports CPUID - _cpuFeatures = feature_flags(); - // Logical processors are only available on P4s and above, - // and only if hyperthreading is available. - _logical_processors_per_package = logical_processor_count(); - _L1_data_cache_line_size = L1_line_size(); - } + if (cpu_family() > 4) { // it supports CPUID + _features = feature_flags(); + // Logical processors are only available on P4s and above, + // and only if hyperthreading is available. + _logical_processors_per_package = logical_processor_count(); + _L1_data_cache_line_size = L1_line_size(); } _supports_cx8 = supports_cmpxchg8(); @@ -524,24 +520,24 @@ void VM_Version::get_processor_features() { // If the OS doesn't support SSE, we can't use this feature even if the HW does if (!os::supports_sse()) - _cpuFeatures &= ~(CPU_SSE|CPU_SSE2|CPU_SSE3|CPU_SSSE3|CPU_SSE4A|CPU_SSE4_1|CPU_SSE4_2); + _features &= ~(CPU_SSE|CPU_SSE2|CPU_SSE3|CPU_SSSE3|CPU_SSE4A|CPU_SSE4_1|CPU_SSE4_2); if (UseSSE < 4) { - _cpuFeatures &= ~CPU_SSE4_1; - _cpuFeatures &= ~CPU_SSE4_2; + _features &= ~CPU_SSE4_1; + _features &= ~CPU_SSE4_2; } if (UseSSE < 3) { - _cpuFeatures &= ~CPU_SSE3; - _cpuFeatures &= ~CPU_SSSE3; - _cpuFeatures &= ~CPU_SSE4A; + _features &= ~CPU_SSE3; + _features &= ~CPU_SSSE3; + _features &= ~CPU_SSE4A; } if (UseSSE < 2) - _cpuFeatures &= ~CPU_SSE2; + _features &= ~CPU_SSE2; if (UseSSE < 1) - _cpuFeatures &= ~CPU_SSE; + _features &= ~CPU_SSE; // first try initial setting and detect what we can support if (UseAVX > 0) { @@ -559,25 +555,25 @@ void VM_Version::get_processor_features() { } if (UseAVX < 3) { - _cpuFeatures &= ~CPU_AVX512F; - _cpuFeatures &= ~CPU_AVX512DQ; - _cpuFeatures &= ~CPU_AVX512CD; - _cpuFeatures &= ~CPU_AVX512BW; - _cpuFeatures &= ~CPU_AVX512VL; + _features &= ~CPU_AVX512F; + _features &= ~CPU_AVX512DQ; + _features &= ~CPU_AVX512CD; + _features &= ~CPU_AVX512BW; + _features &= ~CPU_AVX512VL; } if (UseAVX < 2) - _cpuFeatures &= ~CPU_AVX2; + _features &= ~CPU_AVX2; if (UseAVX < 1) - _cpuFeatures &= ~CPU_AVX; + _features &= ~CPU_AVX; if (!UseAES && !FLAG_IS_DEFAULT(UseAES)) - _cpuFeatures &= ~CPU_AES; + _features &= ~CPU_AES; if (logical_processors_per_package() == 1) { // HT processor could be installed on a system which doesn't support HT. - _cpuFeatures &= ~CPU_HT; + _features &= ~CPU_HT; } char buf[256]; @@ -613,7 +609,7 @@ void VM_Version::get_processor_features() { (supports_bmi2() ? ", bmi2" : ""), (supports_adx() ? ", adx" : ""), (supports_evex() ? ", evex" : "")); - _features_str = os::strdup(buf); + _features_string = os::strdup(buf); // UseSSE is set to the smaller of what hardware supports and what // the command line requires. I.e., you cannot set UseSSE to 2 on @@ -652,6 +648,28 @@ void VM_Version::get_processor_features() { } FLAG_SET_DEFAULT(UseAESIntrinsics, false); } + + // --AES-CTR begins-- + if (!UseAESIntrinsics) { + if (UseAESCTRIntrinsics && !FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) { + warning("AES-CTR intrinsics require UseAESIntrinsics flag to be enabled. Intrinsics will be disabled."); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + } else { + if(supports_sse4_1() && UseSSE >= 4) { + if (FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) { + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, true); + } + } else { + // The AES-CTR intrinsic stubs require AES instruction support (of course) + // but also require sse4.1 mode or higher for instructions it use. + if (UseAESCTRIntrinsics && !FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) { + warning("X86 AES-CTR intrinsics require SSE4.1 instructions or higher. Intrinsics will be disabled."); + } + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + } + // --AES-CTR ends-- } } else if (UseAES || UseAESIntrinsics) { if (UseAES && !FLAG_IS_DEFAULT(UseAES)) { @@ -662,6 +680,10 @@ void VM_Version::get_processor_features() { warning("AES intrinsics are not available on this CPU"); FLAG_SET_DEFAULT(UseAESIntrinsics, false); } + if (UseAESCTRIntrinsics && !FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) { + warning("AES-CTR intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } } // Use CLMUL instructions if available. @@ -685,6 +707,16 @@ void VM_Version::get_processor_features() { FLAG_SET_DEFAULT(UseCRC32Intrinsics, false); } + if (UseAESIntrinsics) { + if (FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) { + UseAESCTRIntrinsics = true; + } + } else if (UseAESCTRIntrinsics) { + if (!FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) + warning("AES/CTR intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false); + } + if (supports_sse4_2()) { if (FLAG_IS_DEFAULT(UseCRC32CIntrinsics)) { UseCRC32CIntrinsics = true; @@ -1041,6 +1073,25 @@ void VM_Version::get_processor_features() { } } +#ifdef _LP64 + if (UseSSE42Intrinsics) { + if (FLAG_IS_DEFAULT(UseVectorizedMismatchIntrinsic)) { + UseVectorizedMismatchIntrinsic = true; + } + } else if (UseVectorizedMismatchIntrinsic) { + if (!FLAG_IS_DEFAULT(UseVectorizedMismatchIntrinsic)) + warning("vectorizedMismatch intrinsics are not available on this CPU"); + FLAG_SET_DEFAULT(UseVectorizedMismatchIntrinsic, false); + } +#else + if (UseVectorizedMismatchIntrinsic) { + if (!FLAG_IS_DEFAULT(UseVectorizedMismatchIntrinsic)) { + warning("vectorizedMismatch intrinsic is not available in 32-bit VM"); + } + FLAG_SET_DEFAULT(UseVectorizedMismatchIntrinsic, false); + } +#endif // _LP64 + // Use count leading zeros count instruction if available. if (supports_lzcnt()) { if (FLAG_IS_DEFAULT(UseCountLeadingZerosInstruction)) { diff --git a/hotspot/src/cpu/x86/vm/vm_version_x86.hpp b/hotspot/src/cpu/x86/vm/vm_version_x86.hpp index 784e8475da9..77550a96a4c 100644 --- a/hotspot/src/cpu/x86/vm/vm_version_x86.hpp +++ b/hotspot/src/cpu/x86/vm/vm_version_x86.hpp @@ -30,7 +30,9 @@ class VM_Version : public Abstract_VM_Version { friend class VMStructs; -public: + friend class JVMCIVMStructs; + + public: // cpuid result register layouts. These are all unions of a uint32_t // (in case anyone wants access to the register as a whole) and a bitfield. @@ -244,14 +246,11 @@ protected: static int _cpu; static int _model; static int _stepping; - static uint64_t _cpuFeatures; // features returned by the "cpuid" instruction - // 0 if this instruction is not available - static const char* _features_str; static address _cpuinfo_segv_addr; // address of instruction which causes SEGV static address _cpuinfo_cont_addr; // address of instruction after the one which causes SEGV - enum { + enum Feature_Flag { CPU_CX8 = (1 << 0), // next bits are from cpuid 1 (EDX) CPU_CMOV = (1 << 1), CPU_FXSR = (1 << 2), @@ -285,11 +284,11 @@ protected: CPU_AVX512ER = (1 << 29), CPU_AVX512CD = (1 << 30), CPU_AVX512BW = (1 << 31) - } cpuFeatureFlags; + }; #define CPU_AVX512VL UCONST64(0x100000000) // EVEX instructions with smaller vector length : enums are limited to 32bit - enum { + enum Extended_Family { // AMD CPU_FAMILY_AMD_11H = 0x11, // Intel @@ -307,7 +306,7 @@ protected: CPU_MODEL_HASWELL_E7 = 0x3f, CPU_MODEL_BROADWELL = 0x3d, CPU_MODEL_SKYLAKE = CPU_MODEL_HASWELL_E3 - } cpuExtendedFamily; + }; // cpuid information block. All info derived from executing cpuid with // various function numbers is stored here. Intel and AMD info is @@ -597,9 +596,9 @@ public: static void set_cpuinfo_cont_addr(address pc) { _cpuinfo_cont_addr = pc; } static address cpuinfo_cont_addr() { return _cpuinfo_cont_addr; } - static void clean_cpuFeatures() { _cpuFeatures = 0; } - static void set_avx_cpuFeatures() { _cpuFeatures = (CPU_SSE | CPU_SSE2 | CPU_AVX); } - static void set_evex_cpuFeatures() { _cpuFeatures = (CPU_AVX512F | CPU_SSE | CPU_SSE2 ); } + static void clean_cpuFeatures() { _features = 0; } + static void set_avx_cpuFeatures() { _features = (CPU_SSE | CPU_SSE2 | CPU_AVX); } + static void set_evex_cpuFeatures() { _features = (CPU_AVX512F | CPU_SSE | CPU_SSE2 ); } // Initialization @@ -687,36 +686,36 @@ public: // // Feature identification // - static bool supports_cpuid() { return _cpuFeatures != 0; } - static bool supports_cmpxchg8() { return (_cpuFeatures & CPU_CX8) != 0; } - static bool supports_cmov() { return (_cpuFeatures & CPU_CMOV) != 0; } - static bool supports_fxsr() { return (_cpuFeatures & CPU_FXSR) != 0; } - static bool supports_ht() { return (_cpuFeatures & CPU_HT) != 0; } - static bool supports_mmx() { return (_cpuFeatures & CPU_MMX) != 0; } - static bool supports_sse() { return (_cpuFeatures & CPU_SSE) != 0; } - static bool supports_sse2() { return (_cpuFeatures & CPU_SSE2) != 0; } - static bool supports_sse3() { return (_cpuFeatures & CPU_SSE3) != 0; } - static bool supports_ssse3() { return (_cpuFeatures & CPU_SSSE3)!= 0; } - static bool supports_sse4_1() { return (_cpuFeatures & CPU_SSE4_1) != 0; } - static bool supports_sse4_2() { return (_cpuFeatures & CPU_SSE4_2) != 0; } - static bool supports_popcnt() { return (_cpuFeatures & CPU_POPCNT) != 0; } - static bool supports_avx() { return (_cpuFeatures & CPU_AVX) != 0; } - static bool supports_avx2() { return (_cpuFeatures & CPU_AVX2) != 0; } - static bool supports_tsc() { return (_cpuFeatures & CPU_TSC) != 0; } - static bool supports_aes() { return (_cpuFeatures & CPU_AES) != 0; } - static bool supports_erms() { return (_cpuFeatures & CPU_ERMS) != 0; } - static bool supports_clmul() { return (_cpuFeatures & CPU_CLMUL) != 0; } - static bool supports_rtm() { return (_cpuFeatures & CPU_RTM) != 0; } - static bool supports_bmi1() { return (_cpuFeatures & CPU_BMI1) != 0; } - static bool supports_bmi2() { return (_cpuFeatures & CPU_BMI2) != 0; } - static bool supports_adx() { return (_cpuFeatures & CPU_ADX) != 0; } - static bool supports_evex() { return (_cpuFeatures & CPU_AVX512F) != 0; } - static bool supports_avx512dq() { return (_cpuFeatures & CPU_AVX512DQ) != 0; } - static bool supports_avx512pf() { return (_cpuFeatures & CPU_AVX512PF) != 0; } - static bool supports_avx512er() { return (_cpuFeatures & CPU_AVX512ER) != 0; } - static bool supports_avx512cd() { return (_cpuFeatures & CPU_AVX512CD) != 0; } - static bool supports_avx512bw() { return (_cpuFeatures & CPU_AVX512BW) != 0; } - static bool supports_avx512vl() { return (_cpuFeatures & CPU_AVX512VL) != 0; } + static bool supports_cpuid() { return _features != 0; } + static bool supports_cmpxchg8() { return (_features & CPU_CX8) != 0; } + static bool supports_cmov() { return (_features & CPU_CMOV) != 0; } + static bool supports_fxsr() { return (_features & CPU_FXSR) != 0; } + static bool supports_ht() { return (_features & CPU_HT) != 0; } + static bool supports_mmx() { return (_features & CPU_MMX) != 0; } + static bool supports_sse() { return (_features & CPU_SSE) != 0; } + static bool supports_sse2() { return (_features & CPU_SSE2) != 0; } + static bool supports_sse3() { return (_features & CPU_SSE3) != 0; } + static bool supports_ssse3() { return (_features & CPU_SSSE3)!= 0; } + static bool supports_sse4_1() { return (_features & CPU_SSE4_1) != 0; } + static bool supports_sse4_2() { return (_features & CPU_SSE4_2) != 0; } + static bool supports_popcnt() { return (_features & CPU_POPCNT) != 0; } + static bool supports_avx() { return (_features & CPU_AVX) != 0; } + static bool supports_avx2() { return (_features & CPU_AVX2) != 0; } + static bool supports_tsc() { return (_features & CPU_TSC) != 0; } + static bool supports_aes() { return (_features & CPU_AES) != 0; } + static bool supports_erms() { return (_features & CPU_ERMS) != 0; } + static bool supports_clmul() { return (_features & CPU_CLMUL) != 0; } + static bool supports_rtm() { return (_features & CPU_RTM) != 0; } + static bool supports_bmi1() { return (_features & CPU_BMI1) != 0; } + static bool supports_bmi2() { return (_features & CPU_BMI2) != 0; } + static bool supports_adx() { return (_features & CPU_ADX) != 0; } + static bool supports_evex() { return (_features & CPU_AVX512F) != 0; } + static bool supports_avx512dq() { return (_features & CPU_AVX512DQ) != 0; } + static bool supports_avx512pf() { return (_features & CPU_AVX512PF) != 0; } + static bool supports_avx512er() { return (_features & CPU_AVX512ER) != 0; } + static bool supports_avx512cd() { return (_features & CPU_AVX512CD) != 0; } + static bool supports_avx512bw() { return (_features & CPU_AVX512BW) != 0; } + static bool supports_avx512vl() { return (_features & CPU_AVX512VL) != 0; } static bool supports_avx512vlbw() { return (supports_avx512bw() && supports_avx512vl()); } static bool supports_avx512novl() { return (supports_evex() && !supports_avx512vl()); } static bool supports_avx512nobw() { return (supports_evex() && !supports_avx512bw()); } @@ -745,17 +744,17 @@ public: } // AMD features - static bool supports_3dnow_prefetch() { return (_cpuFeatures & CPU_3DNOW_PREFETCH) != 0; } + static bool supports_3dnow_prefetch() { return (_features & CPU_3DNOW_PREFETCH) != 0; } static bool supports_mmx_ext() { return is_amd() && _cpuid_info.ext_cpuid1_edx.bits.mmx_amd != 0; } - static bool supports_lzcnt() { return (_cpuFeatures & CPU_LZCNT) != 0; } - static bool supports_sse4a() { return (_cpuFeatures & CPU_SSE4A) != 0; } + static bool supports_lzcnt() { return (_features & CPU_LZCNT) != 0; } + static bool supports_sse4a() { return (_features & CPU_SSE4A) != 0; } static bool is_amd_Barcelona() { return is_amd() && extended_cpu_family() == CPU_FAMILY_AMD_11H; } // Intel and AMD newer cores support fast timestamps well static bool supports_tscinv_bit() { - return (_cpuFeatures & CPU_TSCINV) != 0; + return (_features & CPU_TSCINV) != 0; } static bool supports_tscinv() { return supports_tscinv_bit() && @@ -769,8 +768,6 @@ public: static bool supports_compare_and_exchange() { return true; } - static const char* cpu_features() { return _features_str; } - static intx allocate_prefetch_distance() { // This method should be called before allocate_prefetch_style(). // diff --git a/hotspot/src/cpu/x86/vm/x86_32.ad b/hotspot/src/cpu/x86/vm/x86_32.ad index 79f5e2b4462..cebd468acb6 100644 --- a/hotspot/src/cpu/x86/vm/x86_32.ad +++ b/hotspot/src/cpu/x86/vm/x86_32.ad @@ -1902,17 +1902,18 @@ encode %{ // who we intended to call. cbuf.set_insts_mark(); $$$emit8$primary; + if (!_method) { emit_d32_reloc(cbuf, ($meth$$method - (int)(cbuf.insts_end()) - 4), - runtime_call_Relocation::spec(), RELOC_IMM32 ); - } else if (_optimized_virtual) { - emit_d32_reloc(cbuf, ($meth$$method - (int)(cbuf.insts_end()) - 4), - opt_virtual_call_Relocation::spec(), RELOC_IMM32 ); + runtime_call_Relocation::spec(), + RELOC_IMM32); } else { + int method_index = resolved_method_index(cbuf); + RelocationHolder rspec = _optimized_virtual ? opt_virtual_call_Relocation::spec(method_index) + : static_call_Relocation::spec(method_index); emit_d32_reloc(cbuf, ($meth$$method - (int)(cbuf.insts_end()) - 4), - static_call_Relocation::spec(), RELOC_IMM32 ); - } - if (_method) { // Emit stub for static call. + rspec, RELOC_DISP32); + // Emit stubs for static call. address stub = CompiledStaticCall::emit_to_interp_stub(cbuf); if (stub == NULL) { ciEnv::current()->record_failure("CodeCache is full"); @@ -1923,7 +1924,7 @@ encode %{ enc_class Java_Dynamic_Call (method meth) %{ // JAVA DYNAMIC CALL MacroAssembler _masm(&cbuf); - __ ic_call((address)$meth$$method); + __ ic_call((address)$meth$$method, resolved_method_index(cbuf)); %} enc_class Java_Compiled_Call (method meth) %{ // JAVA COMPILED CALL @@ -9884,39 +9885,6 @@ instruct sqrtDPR_reg(regDPR dst, regDPR src) %{ ins_pipe( pipe_slow ); %} -instruct powDPR_reg(regDPR X, regDPR1 Y, eAXRegI rax, eDXRegI rdx, eCXRegI rcx, eFlagsReg cr) %{ - predicate (UseSSE<=1); - match(Set Y (PowD X Y)); // Raise X to the Yth power - effect(KILL rax, KILL rdx, KILL rcx, KILL cr); - format %{ "fast_pow $X $Y -> $Y // KILL $rax, $rcx, $rdx" %} - ins_encode %{ - __ subptr(rsp, 8); - __ fld_s($X$$reg - 1); - __ fast_pow(); - __ addptr(rsp, 8); - %} - ins_pipe( pipe_slow ); -%} - -instruct powD_reg(regD dst, regD src0, regD src1, eAXRegI rax, eDXRegI rdx, eCXRegI rcx, eFlagsReg cr) %{ - predicate (UseSSE>=2); - match(Set dst (PowD src0 src1)); // Raise src0 to the src1'th power - effect(KILL rax, KILL rdx, KILL rcx, KILL cr); - format %{ "fast_pow $src0 $src1 -> $dst // KILL $rax, $rcx, $rdx" %} - ins_encode %{ - __ subptr(rsp, 8); - __ movdbl(Address(rsp, 0), $src1$$XMMRegister); - __ fld_d(Address(rsp, 0)); - __ movdbl(Address(rsp, 0), $src0$$XMMRegister); - __ fld_d(Address(rsp, 0)); - __ fast_pow(); - __ fstp_d(Address(rsp, 0)); - __ movdbl($dst$$XMMRegister, Address(rsp, 0)); - __ addptr(rsp, 8); - %} - ins_pipe( pipe_slow ); -%} - instruct log10DPR_reg(regDPR1 dst, regDPR1 src) %{ predicate (UseSSE<=1); // The source Double operand on FPU stack @@ -11508,7 +11476,7 @@ instruct string_equals(eDIRegP str1, eSIRegP str2, eCXRegI cnt, eAXRegI result, __ arrays_equals(false, $str1$$Register, $str2$$Register, $cnt$$Register, $result$$Register, $tmp3$$Register, $tmp1$$XMMRegister, $tmp2$$XMMRegister, false /* char */); - %} + %} ins_pipe( pipe_slow ); %} diff --git a/hotspot/src/cpu/x86/vm/x86_64.ad b/hotspot/src/cpu/x86/vm/x86_64.ad index 8a3cd428499..9def843e07e 100644 --- a/hotspot/src/cpu/x86/vm/x86_64.ad +++ b/hotspot/src/cpu/x86/vm/x86_64.ad @@ -2125,22 +2125,15 @@ encode %{ $$$emit8$primary; if (!_method) { - emit_d32_reloc(cbuf, - (int) ($meth$$method - ((intptr_t) cbuf.insts_end()) - 4), + emit_d32_reloc(cbuf, (int) ($meth$$method - ((intptr_t) cbuf.insts_end()) - 4), runtime_call_Relocation::spec(), RELOC_DISP32); - } else if (_optimized_virtual) { - emit_d32_reloc(cbuf, - (int) ($meth$$method - ((intptr_t) cbuf.insts_end()) - 4), - opt_virtual_call_Relocation::spec(), - RELOC_DISP32); } else { - emit_d32_reloc(cbuf, - (int) ($meth$$method - ((intptr_t) cbuf.insts_end()) - 4), - static_call_Relocation::spec(), - RELOC_DISP32); - } - if (_method) { + int method_index = resolved_method_index(cbuf); + RelocationHolder rspec = _optimized_virtual ? opt_virtual_call_Relocation::spec(method_index) + : static_call_Relocation::spec(method_index); + emit_d32_reloc(cbuf, (int) ($meth$$method - ((intptr_t) cbuf.insts_end()) - 4), + rspec, RELOC_DISP32); // Emit stubs for static call. address mark = cbuf.insts_mark(); address stub = CompiledStaticCall::emit_to_interp_stub(cbuf, mark); @@ -2153,7 +2146,7 @@ encode %{ enc_class Java_Dynamic_Call(method meth) %{ MacroAssembler _masm(&cbuf); - __ ic_call((address)$meth$$method); + __ ic_call((address)$meth$$method, resolved_method_index(cbuf)); %} enc_class Java_Compiled_Call(method meth) @@ -9871,24 +9864,6 @@ instruct log10D_reg(regD dst) %{ ins_pipe( pipe_slow ); %} -instruct powD_reg(regD dst, regD src0, regD src1, rax_RegI rax, rdx_RegI rdx, rcx_RegI rcx, rFlagsReg cr) %{ - match(Set dst (PowD src0 src1)); // Raise src0 to the src1'th power - effect(KILL rax, KILL rdx, KILL rcx, KILL cr); - format %{ "fast_pow $src0 $src1 -> $dst // KILL $rax, $rcx, $rdx" %} - ins_encode %{ - __ subptr(rsp, 8); - __ movdbl(Address(rsp, 0), $src1$$XMMRegister); - __ fld_d(Address(rsp, 0)); - __ movdbl(Address(rsp, 0), $src0$$XMMRegister); - __ fld_d(Address(rsp, 0)); - __ fast_pow(); - __ fstp_d(Address(rsp, 0)); - __ movdbl($dst$$XMMRegister, Address(rsp, 0)); - __ addptr(rsp, 8); - %} - ins_pipe( pipe_slow ); -%} - //----------Arithmetic Conversion Instructions--------------------------------- instruct roundFloat_nop(regF dst) diff --git a/hotspot/src/cpu/zero/vm/globalDefinitions_zero.hpp b/hotspot/src/cpu/zero/vm/globalDefinitions_zero.hpp index 5956fe1cfcc..144c47e3cb9 100644 --- a/hotspot/src/cpu/zero/vm/globalDefinitions_zero.hpp +++ b/hotspot/src/cpu/zero/vm/globalDefinitions_zero.hpp @@ -28,4 +28,8 @@ #include +// Indicates whether the C calling conventions require that +// 32-bit integer argument values are extended to 64 bits. +const bool CCallingConventionRequiresIntsAsLongs = false; + #endif // CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP diff --git a/hotspot/src/cpu/zero/vm/vm_version_zero.hpp b/hotspot/src/cpu/zero/vm/vm_version_zero.hpp index 68a29df3a60..51b9c5b3035 100644 --- a/hotspot/src/cpu/zero/vm/vm_version_zero.hpp +++ b/hotspot/src/cpu/zero/vm/vm_version_zero.hpp @@ -31,9 +31,6 @@ class VM_Version : public Abstract_VM_Version { public: - static const char* cpu_features() { - return ""; - } static void initialize(); }; diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64.java new file mode 100644 index 00000000000..0038c161b1b --- /dev/null +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64.java @@ -0,0 +1,235 @@ +/* + * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package jdk.vm.ci.aarch64; + +import java.nio.ByteOrder; +import java.util.EnumSet; + +import jdk.vm.ci.code.Architecture; +import jdk.vm.ci.code.Register; +import jdk.vm.ci.code.Register.RegisterCategory; +import jdk.vm.ci.meta.JavaKind; +import jdk.vm.ci.meta.PlatformKind; + +/** + * Represents the AArch64 architecture. + */ +public class AArch64 extends Architecture { + + public static final RegisterCategory CPU = new RegisterCategory("CPU"); + + // General purpose CPU registers + public static final Register r0 = new Register(0, 0, "r0", CPU); + public static final Register r1 = new Register(1, 1, "r1", CPU); + public static final Register r2 = new Register(2, 2, "r2", CPU); + public static final Register r3 = new Register(3, 3, "r3", CPU); + public static final Register r4 = new Register(4, 4, "r4", CPU); + public static final Register r5 = new Register(5, 5, "r5", CPU); + public static final Register r6 = new Register(6, 6, "r6", CPU); + public static final Register r7 = new Register(7, 7, "r7", CPU); + public static final Register r8 = new Register(8, 8, "r8", CPU); + public static final Register r9 = new Register(9, 9, "r9", CPU); + public static final Register r10 = new Register(10, 10, "r10", CPU); + public static final Register r11 = new Register(11, 11, "r11", CPU); + public static final Register r12 = new Register(12, 12, "r12", CPU); + public static final Register r13 = new Register(13, 13, "r13", CPU); + public static final Register r14 = new Register(14, 14, "r14", CPU); + public static final Register r15 = new Register(15, 15, "r15", CPU); + public static final Register r16 = new Register(16, 16, "r16", CPU); + public static final Register r17 = new Register(17, 17, "r17", CPU); + public static final Register r18 = new Register(18, 18, "r18", CPU); + public static final Register r19 = new Register(19, 19, "r19", CPU); + public static final Register r20 = new Register(20, 20, "r20", CPU); + public static final Register r21 = new Register(21, 21, "r21", CPU); + public static final Register r22 = new Register(22, 22, "r22", CPU); + public static final Register r23 = new Register(23, 23, "r23", CPU); + public static final Register r24 = new Register(24, 24, "r24", CPU); + public static final Register r25 = new Register(25, 25, "r25", CPU); + public static final Register r26 = new Register(26, 26, "r26", CPU); + public static final Register r27 = new Register(27, 27, "r27", CPU); + public static final Register r28 = new Register(28, 28, "r28", CPU); + public static final Register r29 = new Register(29, 29, "r29", CPU); + public static final Register r30 = new Register(30, 30, "r30", CPU); + public static final Register r31 = new Register(31, 31, "r31", CPU); + + public static final Register lr = r30; + public static final Register zr = r31; + public static final Register sp = r31; + + public static final Register[] cpuRegisters = { + // @formatter:off + r0, r1, r2, r3, r4, r5, r6, r7, + r8, r9, r10, r11, r12, r13, r14, r15, + r16, r17, r18, r19, r20, r21, r22, r23, + r24, r25, r26, r27, r28, r29, r30, r31 + // @formatter:on + }; + + public static final RegisterCategory SIMD = new RegisterCategory("SIMD"); + + // Simd registers + public static final Register v0 = new Register(32, 0, "v0", SIMD); + public static final Register v1 = new Register(33, 1, "v1", SIMD); + public static final Register v2 = new Register(34, 2, "v2", SIMD); + public static final Register v3 = new Register(35, 3, "v3", SIMD); + public static final Register v4 = new Register(36, 4, "v4", SIMD); + public static final Register v5 = new Register(37, 5, "v5", SIMD); + public static final Register v6 = new Register(38, 6, "v6", SIMD); + public static final Register v7 = new Register(39, 7, "v7", SIMD); + public static final Register v8 = new Register(40, 8, "v8", SIMD); + public static final Register v9 = new Register(41, 9, "v9", SIMD); + public static final Register v10 = new Register(42, 10, "v10", SIMD); + public static final Register v11 = new Register(43, 11, "v11", SIMD); + public static final Register v12 = new Register(44, 12, "v12", SIMD); + public static final Register v13 = new Register(45, 13, "v13", SIMD); + public static final Register v14 = new Register(46, 14, "v14", SIMD); + public static final Register v15 = new Register(47, 15, "v15", SIMD); + public static final Register v16 = new Register(48, 16, "v16", SIMD); + public static final Register v17 = new Register(49, 17, "v17", SIMD); + public static final Register v18 = new Register(50, 18, "v18", SIMD); + public static final Register v19 = new Register(51, 19, "v19", SIMD); + public static final Register v20 = new Register(52, 20, "v20", SIMD); + public static final Register v21 = new Register(53, 21, "v21", SIMD); + public static final Register v22 = new Register(54, 22, "v22", SIMD); + public static final Register v23 = new Register(55, 23, "v23", SIMD); + public static final Register v24 = new Register(56, 24, "v24", SIMD); + public static final Register v25 = new Register(57, 25, "v25", SIMD); + public static final Register v26 = new Register(58, 26, "v26", SIMD); + public static final Register v27 = new Register(59, 27, "v27", SIMD); + public static final Register v28 = new Register(60, 28, "v28", SIMD); + public static final Register v29 = new Register(61, 29, "v29", SIMD); + public static final Register v30 = new Register(62, 30, "v30", SIMD); + public static final Register v31 = new Register(63, 31, "v31", SIMD); + + public static final Register[] simdRegisters = { + // @formatter:off + v0, v1, v2, v3, v4, v5, v6, v7, + v8, v9, v10, v11, v12, v13, v14, v15, + v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31 + // @formatter:on + }; + + public static final Register[] allRegisters = { + // @formatter:off + r0, r1, r2, r3, r4, r5, r6, r7, + r8, r9, r10, r11, r12, r13, r14, r15, + r16, r17, r18, r19, r20, r21, r22, r23, + r24, r25, r26, r27, r28, r29, r30, r31, + + v0, v1, v2, v3, v4, v5, v6, v7, + v8, v9, v10, v11, v12, v13, v14, v15, + v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31 + // @formatter:on + }; + + /** + * Basic set of CPU features mirroring what is returned from the cpuid instruction. See: + * {@code VM_Version::cpuFeatureFlags}. + */ + public static enum CPUFeature { + FP, + ASIMD, + EVTSTRM, + AES, + PMULL, + SHA1, + SHA2, + CRC32, + A53MAC, + DMB_ATOMICS + } + + private final EnumSet features; + + /** + * Set of flags to control code emission. + */ + public static enum Flag { + UseBarriersForVolatile, + UseCRC32, + UseNeon + } + + private final EnumSet flags; + + public AArch64(EnumSet features, EnumSet flags) { + super("aarch64", AArch64Kind.QWORD, ByteOrder.LITTLE_ENDIAN, true, allRegisters, 0, 0, 0); + this.features = features; + this.flags = flags; + } + + public EnumSet getFeatures() { + return features; + } + + public EnumSet getFlags() { + return flags; + } + + @Override + public PlatformKind getPlatformKind(JavaKind javaKind) { + switch (javaKind) { + case Boolean: + case Byte: + return AArch64Kind.BYTE; + case Short: + case Char: + return AArch64Kind.WORD; + case Int: + return AArch64Kind.DWORD; + case Long: + case Object: + return AArch64Kind.QWORD; + case Float: + return AArch64Kind.SINGLE; + case Double: + return AArch64Kind.DOUBLE; + default: + return null; + } + } + + @Override + public boolean canStoreValue(RegisterCategory category, PlatformKind platformKind) { + AArch64Kind kind = (AArch64Kind) platformKind; + if (kind.isInteger()) { + return category.equals(CPU); + } else if (kind.isSIMD()) { + return category.equals(SIMD); + } + return false; + } + + @Override + public AArch64Kind getLargestStorableKind(RegisterCategory category) { + if (category.equals(CPU)) { + return AArch64Kind.QWORD; + } else if (category.equals(SIMD)) { + return AArch64Kind.V128_QWORD; + } else { + return null; + } + } +} diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64Kind.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64Kind.java new file mode 100644 index 00000000000..1802ef7eab2 --- /dev/null +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.aarch64/src/jdk/vm/ci/aarch64/AArch64Kind.java @@ -0,0 +1,171 @@ +/* + * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package jdk.vm.ci.aarch64; + +import jdk.vm.ci.meta.PlatformKind; + +public enum AArch64Kind implements PlatformKind { + + // scalar + BYTE(1), + WORD(2), + DWORD(4), + QWORD(8), + SINGLE(4), + DOUBLE(8), + + // SIMD + V32_BYTE(4, BYTE), + V32_WORD(4, WORD), + V64_BYTE(8, BYTE), + V64_WORD(8, WORD), + V64_DWORD(8, DWORD), + V128_BYTE(16, BYTE), + V128_WORD(16, WORD), + V128_DWORD(16, DWORD), + V128_QWORD(16, QWORD), + V128_SINGLE(16, SINGLE), + V128_DOUBLE(16, DOUBLE), + + MASK8(1), + MASK16(2), + MASK32(4), + MASK64(8); + + private final int size; + private final int vectorLength; + + private final AArch64Kind scalar; + private final EnumKey key = new EnumKey<>(this); + + private AArch64Kind(int size) { + this.size = size; + this.scalar = this; + this.vectorLength = 1; + } + + private AArch64Kind(int size, AArch64Kind scalar) { + this.size = size; + this.scalar = scalar; + + assert size % scalar.size == 0; + this.vectorLength = size / scalar.size; + } + + public AArch64Kind getScalar() { + return scalar; + } + + public int getSizeInBytes() { + return size; + } + + public int getVectorLength() { + return vectorLength; + } + + public Key getKey() { + return key; + } + + public boolean isInteger() { + switch (this) { + case BYTE: + case WORD: + case DWORD: + case QWORD: + return true; + default: + return false; + } + } + + public boolean isSIMD() { + switch (this) { + case SINGLE: + case DOUBLE: + case V32_BYTE: + case V32_WORD: + case V64_BYTE: + case V64_WORD: + case V64_DWORD: + case V128_BYTE: + case V128_WORD: + case V128_DWORD: + case V128_QWORD: + case V128_SINGLE: + case V128_DOUBLE: + return true; + default: + return false; + } + } + + public boolean isMask() { + switch (this) { + case MASK8: + case MASK16: + case MASK32: + case MASK64: + return true; + default: + return false; + } + } + + public char getTypeChar() { + switch (this) { + case BYTE: + return 'b'; + case WORD: + return 'w'; + case DWORD: + return 'd'; + case QWORD: + return 'q'; + case SINGLE: + return 'S'; + case DOUBLE: + return 'D'; + case V32_BYTE: + case V32_WORD: + case V64_BYTE: + case V64_WORD: + case V64_DWORD: + case V128_BYTE: + case V128_WORD: + case V128_DWORD: + case V128_QWORD: + case V128_SINGLE: + case V128_DOUBLE: + return 'v'; + case MASK8: + case MASK16: + case MASK32: + case MASK64: + return 'k'; + default: + return '-'; + } + } +} diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotJVMCIBackendFactory.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotJVMCIBackendFactory.java new file mode 100644 index 00000000000..6981820018b --- /dev/null +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotJVMCIBackendFactory.java @@ -0,0 +1,131 @@ +/* + * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package jdk.vm.ci.hotspot.aarch64; + +import static jdk.vm.ci.inittimer.InitTimer.timer; + +import java.util.EnumSet; + +import jdk.vm.ci.aarch64.AArch64; +import jdk.vm.ci.code.Architecture; +import jdk.vm.ci.code.RegisterConfig; +import jdk.vm.ci.code.TargetDescription; +import jdk.vm.ci.code.stack.StackIntrospection; +import jdk.vm.ci.hotspot.HotSpotCodeCacheProvider; +import jdk.vm.ci.hotspot.HotSpotConstantReflectionProvider; +import jdk.vm.ci.hotspot.HotSpotJVMCIBackendFactory; +import jdk.vm.ci.hotspot.HotSpotJVMCIRuntimeProvider; +import jdk.vm.ci.hotspot.HotSpotMetaAccessProvider; +import jdk.vm.ci.hotspot.HotSpotStackIntrospection; +import jdk.vm.ci.hotspot.HotSpotVMConfig; +import jdk.vm.ci.inittimer.InitTimer; +import jdk.vm.ci.meta.ConstantReflectionProvider; +import jdk.vm.ci.runtime.JVMCIBackend; +import jdk.vm.ci.service.ServiceProvider; + +@ServiceProvider(HotSpotJVMCIBackendFactory.class) +public class AArch64HotSpotJVMCIBackendFactory implements HotSpotJVMCIBackendFactory { + + protected EnumSet computeFeatures(@SuppressWarnings("unused") HotSpotVMConfig config) { + // Configure the feature set using the HotSpot flag settings. + EnumSet features = EnumSet.noneOf(AArch64.CPUFeature.class); + return features; + } + + protected EnumSet computeFlags(@SuppressWarnings("unused") HotSpotVMConfig config) { + EnumSet flags = EnumSet.noneOf(AArch64.Flag.class); + return flags; + } + + protected TargetDescription createTarget(HotSpotVMConfig config) { + final int stackFrameAlignment = 16; + final int implicitNullCheckLimit = 4096; + final boolean inlineObjects = true; + Architecture arch = new AArch64(computeFeatures(config), computeFlags(config)); + return new TargetDescription(arch, true, stackFrameAlignment, implicitNullCheckLimit, inlineObjects); + } + + protected HotSpotConstantReflectionProvider createConstantReflection(HotSpotJVMCIRuntimeProvider runtime) { + return new HotSpotConstantReflectionProvider(runtime); + } + + protected RegisterConfig createRegisterConfig(HotSpotJVMCIRuntimeProvider runtime, TargetDescription target) { + return new AArch64HotSpotRegisterConfig(target.arch, runtime.getConfig()); + } + + protected HotSpotCodeCacheProvider createCodeCache(HotSpotJVMCIRuntimeProvider runtime, TargetDescription target, RegisterConfig regConfig) { + return new HotSpotCodeCacheProvider(runtime, runtime.getConfig(), target, regConfig); + } + + protected HotSpotMetaAccessProvider createMetaAccess(HotSpotJVMCIRuntimeProvider runtime) { + return new HotSpotMetaAccessProvider(runtime); + } + + @Override + public String getArchitecture() { + return "aarch64"; + } + + @Override + public String toString() { + return "JVMCIBackend:" + getArchitecture(); + } + + @SuppressWarnings("try") + public JVMCIBackend createJVMCIBackend(HotSpotJVMCIRuntimeProvider runtime, JVMCIBackend host) { + + assert host == null; + TargetDescription target = createTarget(runtime.getConfig()); + + RegisterConfig regConfig; + HotSpotCodeCacheProvider codeCache; + ConstantReflectionProvider constantReflection; + HotSpotMetaAccessProvider metaAccess; + StackIntrospection stackIntrospection; + try (InitTimer t = timer("create providers")) { + try (InitTimer rt = timer("create MetaAccess provider")) { + metaAccess = createMetaAccess(runtime); + } + try (InitTimer rt = timer("create RegisterConfig")) { + regConfig = createRegisterConfig(runtime, target); + } + try (InitTimer rt = timer("create CodeCache provider")) { + codeCache = createCodeCache(runtime, target, regConfig); + } + try (InitTimer rt = timer("create ConstantReflection provider")) { + constantReflection = createConstantReflection(runtime); + } + try (InitTimer rt = timer("create StackIntrospection provider")) { + stackIntrospection = new HotSpotStackIntrospection(runtime); + } + } + try (InitTimer rt = timer("instantiate backend")) { + return createBackend(metaAccess, codeCache, constantReflection, stackIntrospection); + } + } + + protected JVMCIBackend createBackend(HotSpotMetaAccessProvider metaAccess, HotSpotCodeCacheProvider codeCache, ConstantReflectionProvider constantReflection, + StackIntrospection stackIntrospection) { + return new JVMCIBackend(metaAccess, codeCache, constantReflection, stackIntrospection); + } +} diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotRegisterConfig.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotRegisterConfig.java new file mode 100644 index 00000000000..9590d2fcda4 --- /dev/null +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.aarch64/src/jdk/vm/ci/hotspot/aarch64/AArch64HotSpotRegisterConfig.java @@ -0,0 +1,298 @@ +/* + * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ +package jdk.vm.ci.hotspot.aarch64; + +import static jdk.vm.ci.aarch64.AArch64.lr; +import static jdk.vm.ci.aarch64.AArch64.r0; +import static jdk.vm.ci.aarch64.AArch64.r1; +import static jdk.vm.ci.aarch64.AArch64.r12; +import static jdk.vm.ci.aarch64.AArch64.r2; +import static jdk.vm.ci.aarch64.AArch64.r27; +import static jdk.vm.ci.aarch64.AArch64.r28; +import static jdk.vm.ci.aarch64.AArch64.r29; +import static jdk.vm.ci.aarch64.AArch64.r3; +import static jdk.vm.ci.aarch64.AArch64.r4; +import static jdk.vm.ci.aarch64.AArch64.r5; +import static jdk.vm.ci.aarch64.AArch64.r6; +import static jdk.vm.ci.aarch64.AArch64.r7; +import static jdk.vm.ci.aarch64.AArch64.r9; +import static jdk.vm.ci.aarch64.AArch64.sp; +import static jdk.vm.ci.aarch64.AArch64.v0; +import static jdk.vm.ci.aarch64.AArch64.v1; +import static jdk.vm.ci.aarch64.AArch64.v2; +import static jdk.vm.ci.aarch64.AArch64.v3; +import static jdk.vm.ci.aarch64.AArch64.v4; +import static jdk.vm.ci.aarch64.AArch64.v5; +import static jdk.vm.ci.aarch64.AArch64.v6; +import static jdk.vm.ci.aarch64.AArch64.v7; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashSet; +import java.util.Set; + +import jdk.vm.ci.aarch64.AArch64; +import jdk.vm.ci.code.Architecture; +import jdk.vm.ci.code.CallingConvention; +import jdk.vm.ci.code.CallingConvention.Type; +import jdk.vm.ci.code.Register; +import jdk.vm.ci.code.RegisterAttributes; +import jdk.vm.ci.code.RegisterConfig; +import jdk.vm.ci.code.StackSlot; +import jdk.vm.ci.code.TargetDescription; +import jdk.vm.ci.common.JVMCIError; +import jdk.vm.ci.hotspot.HotSpotVMConfig; +import jdk.vm.ci.meta.AllocatableValue; +import jdk.vm.ci.meta.JavaKind; +import jdk.vm.ci.meta.JavaType; +import jdk.vm.ci.meta.LIRKind; +import jdk.vm.ci.meta.PlatformKind; +import jdk.vm.ci.meta.Value; + +public class AArch64HotSpotRegisterConfig implements RegisterConfig { + + private final Architecture architecture; + + private final Register[] allocatable; + + private final int maxFrameSize; + + /** + * The caller saved registers always include all parameter registers. + */ + private final Register[] callerSaved; + + private final boolean allAllocatableAreCallerSaved; + + private final RegisterAttributes[] attributesMap; + + public int getMaximumFrameSize() { + return maxFrameSize; + } + + @Override + public Register[] getAllocatableRegisters() { + return allocatable.clone(); + } + + public Register[] filterAllocatableRegisters(PlatformKind kind, Register[] registers) { + ArrayList list = new ArrayList<>(); + for (Register reg : registers) { + if (architecture.canStoreValue(reg.getRegisterCategory(), kind)) { + list.add(reg); + } + } + + Register[] ret = list.toArray(new Register[list.size()]); + return ret; + } + + @Override + public RegisterAttributes[] getAttributesMap() { + return attributesMap.clone(); + } + + private final Register[] javaGeneralParameterRegisters = {r1, r2, r3, r4, r5, r6, r7, r0}; + private final Register[] nativeGeneralParameterRegisters = {r0, r1, r2, r3, r4, r5, r6, r7}; + private final Register[] simdParameterRegisters = {v0, v1, v2, v3, v4, v5, v6, v7}; + + public static final Register inlineCacheRegister = r9; + + /** + * Vtable stubs expect the metaspace Method in r12. + */ + public static final Register metaspaceMethodRegister = r12; + + public static final Register heapBaseRegister = r27; + public static final Register threadRegister = r28; + public static final Register fp = r29; + + private static Register[] initAllocatable(Architecture arch, boolean reserveForHeapBase) { + Register[] allRegisters = arch.getAvailableValueRegisters(); + Register[] registers = new Register[allRegisters.length - (reserveForHeapBase ? 5 : 4)]; + + int idx = 0; + for (Register reg : allRegisters) { + if (reg.equals(threadRegister) || reg.equals(fp) || reg.equals(lr) || reg.equals(sp)) { + // skip thread register, frame pointer, link register and stack pointer + continue; + } + if (reserveForHeapBase && reg.equals(heapBaseRegister)) { + // skip heap base register + continue; + } + + registers[idx++] = reg; + } + + assert idx == registers.length; + return registers; + } + + public AArch64HotSpotRegisterConfig(Architecture architecture, HotSpotVMConfig config) { + this(architecture, config, initAllocatable(architecture, config.useCompressedOops)); + assert callerSaved.length >= allocatable.length; + } + + public AArch64HotSpotRegisterConfig(Architecture architecture, HotSpotVMConfig config, Register[] allocatable) { + this.architecture = architecture; + this.maxFrameSize = config.maxFrameSize; + + this.allocatable = allocatable.clone(); + Set callerSaveSet = new HashSet<>(); + Collections.addAll(callerSaveSet, allocatable); + Collections.addAll(callerSaveSet, simdParameterRegisters); + Collections.addAll(callerSaveSet, javaGeneralParameterRegisters); + Collections.addAll(callerSaveSet, nativeGeneralParameterRegisters); + callerSaved = callerSaveSet.toArray(new Register[callerSaveSet.size()]); + + allAllocatableAreCallerSaved = true; + attributesMap = RegisterAttributes.createMap(this, AArch64.allRegisters); + } + + @Override + public Register[] getCallerSaveRegisters() { + return callerSaved; + } + + public Register[] getCalleeSaveRegisters() { + return null; + } + + @Override + public boolean areAllAllocatableRegistersCallerSaved() { + return allAllocatableAreCallerSaved; + } + + @Override + public Register getRegisterForRole(int index) { + throw new UnsupportedOperationException(); + } + + @Override + public CallingConvention getCallingConvention(Type type, JavaType returnType, JavaType[] parameterTypes, TargetDescription target, boolean stackOnly) { + if (type == Type.NativeCall) { + return callingConvention(nativeGeneralParameterRegisters, returnType, parameterTypes, type, target, stackOnly); + } + // On x64, parameter locations are the same whether viewed + // from the caller or callee perspective + return callingConvention(javaGeneralParameterRegisters, returnType, parameterTypes, type, target, stackOnly); + } + + public Register[] getCallingConventionRegisters(Type type, JavaKind kind) { + switch (kind) { + case Boolean: + case Byte: + case Short: + case Char: + case Int: + case Long: + case Object: + return type == Type.NativeCall ? nativeGeneralParameterRegisters : javaGeneralParameterRegisters; + case Float: + case Double: + return simdParameterRegisters; + default: + throw JVMCIError.shouldNotReachHere(); + } + } + + private CallingConvention callingConvention(Register[] generalParameterRegisters, JavaType returnType, JavaType[] parameterTypes, Type type, TargetDescription target, boolean stackOnly) { + AllocatableValue[] locations = new AllocatableValue[parameterTypes.length]; + + int currentGeneral = 0; + int currentSIMD = 0; + int currentStackOffset = 0; + + for (int i = 0; i < parameterTypes.length; i++) { + final JavaKind kind = parameterTypes[i].getJavaKind().getStackKind(); + + switch (kind) { + case Byte: + case Boolean: + case Short: + case Char: + case Int: + case Long: + case Object: + if (!stackOnly && currentGeneral < generalParameterRegisters.length) { + Register register = generalParameterRegisters[currentGeneral++]; + locations[i] = register.asValue(target.getLIRKind(kind)); + } + break; + case Float: + case Double: + if (!stackOnly && currentSIMD < simdParameterRegisters.length) { + Register register = simdParameterRegisters[currentSIMD++]; + locations[i] = register.asValue(target.getLIRKind(kind)); + } + break; + default: + throw JVMCIError.shouldNotReachHere(); + } + + if (locations[i] == null) { + LIRKind lirKind = target.getLIRKind(kind); + locations[i] = StackSlot.get(lirKind, currentStackOffset, !type.out); + currentStackOffset += Math.max(lirKind.getPlatformKind().getSizeInBytes(), target.wordSize); + } + } + + JavaKind returnKind = returnType == null ? JavaKind.Void : returnType.getJavaKind(); + AllocatableValue returnLocation = returnKind == JavaKind.Void ? Value.ILLEGAL : getReturnRegister(returnKind).asValue(target.getLIRKind(returnKind.getStackKind())); + return new CallingConvention(currentStackOffset, returnLocation, locations); + } + + @Override + public Register getReturnRegister(JavaKind kind) { + switch (kind) { + case Boolean: + case Byte: + case Char: + case Short: + case Int: + case Long: + case Object: + return r0; + case Float: + case Double: + return v0; + case Void: + case Illegal: + return null; + default: + throw new UnsupportedOperationException("no return register for type " + kind); + } + } + + @Override + public Register getFrameRegister() { + return sp; + } + + @Override + public String toString() { + return String.format("Allocatable: " + Arrays.toString(getAllocatableRegisters()) + "%n" + "CallerSave: " + Arrays.toString(getCallerSaveRegisters()) + "%n"); + } +} diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.amd64/src/jdk/vm/ci/hotspot/amd64/AMD64HotSpotJVMCIBackendFactory.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.amd64/src/jdk/vm/ci/hotspot/amd64/AMD64HotSpotJVMCIBackendFactory.java index d5d31ea86e5..fb6eba20f92 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.amd64/src/jdk/vm/ci/hotspot/amd64/AMD64HotSpotJVMCIBackendFactory.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.amd64/src/jdk/vm/ci/hotspot/amd64/AMD64HotSpotJVMCIBackendFactory.java @@ -49,79 +49,79 @@ public class AMD64HotSpotJVMCIBackendFactory implements HotSpotJVMCIBackendFacto protected EnumSet computeFeatures(HotSpotVMConfig config) { // Configure the feature set using the HotSpot flag settings. EnumSet features = EnumSet.noneOf(AMD64.CPUFeature.class); - if ((config.x86CPUFeatures & config.cpu3DNOWPREFETCH) != 0) { + if ((config.vmVersionFeatures & config.amd643DNOWPREFETCH) != 0) { features.add(AMD64.CPUFeature.AMD_3DNOW_PREFETCH); } assert config.useSSE >= 2 : "minimum config for x64"; features.add(AMD64.CPUFeature.SSE); features.add(AMD64.CPUFeature.SSE2); - if ((config.x86CPUFeatures & config.cpuSSE3) != 0) { + if ((config.vmVersionFeatures & config.amd64SSE3) != 0) { features.add(AMD64.CPUFeature.SSE3); } - if ((config.x86CPUFeatures & config.cpuSSSE3) != 0) { + if ((config.vmVersionFeatures & config.amd64SSSE3) != 0) { features.add(AMD64.CPUFeature.SSSE3); } - if ((config.x86CPUFeatures & config.cpuSSE4A) != 0) { + if ((config.vmVersionFeatures & config.amd64SSE4A) != 0) { features.add(AMD64.CPUFeature.SSE4A); } - if ((config.x86CPUFeatures & config.cpuSSE41) != 0) { + if ((config.vmVersionFeatures & config.amd64SSE41) != 0) { features.add(AMD64.CPUFeature.SSE4_1); } - if ((config.x86CPUFeatures & config.cpuSSE42) != 0) { + if ((config.vmVersionFeatures & config.amd64SSE42) != 0) { features.add(AMD64.CPUFeature.SSE4_2); } - if ((config.x86CPUFeatures & config.cpuPOPCNT) != 0) { + if ((config.vmVersionFeatures & config.amd64POPCNT) != 0) { features.add(AMD64.CPUFeature.POPCNT); } - if ((config.x86CPUFeatures & config.cpuLZCNT) != 0) { + if ((config.vmVersionFeatures & config.amd64LZCNT) != 0) { features.add(AMD64.CPUFeature.LZCNT); } - if ((config.x86CPUFeatures & config.cpuERMS) != 0) { + if ((config.vmVersionFeatures & config.amd64ERMS) != 0) { features.add(AMD64.CPUFeature.ERMS); } - if ((config.x86CPUFeatures & config.cpuAVX) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX) != 0) { features.add(AMD64.CPUFeature.AVX); } - if ((config.x86CPUFeatures & config.cpuAVX2) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX2) != 0) { features.add(AMD64.CPUFeature.AVX2); } - if ((config.x86CPUFeatures & config.cpuAES) != 0) { + if ((config.vmVersionFeatures & config.amd64AES) != 0) { features.add(AMD64.CPUFeature.AES); } - if ((config.x86CPUFeatures & config.cpu3DNOWPREFETCH) != 0) { + if ((config.vmVersionFeatures & config.amd643DNOWPREFETCH) != 0) { features.add(AMD64.CPUFeature.AMD_3DNOW_PREFETCH); } - if ((config.x86CPUFeatures & config.cpuBMI1) != 0) { + if ((config.vmVersionFeatures & config.amd64BMI1) != 0) { features.add(AMD64.CPUFeature.BMI1); } - if ((config.x86CPUFeatures & config.cpuBMI2) != 0) { + if ((config.vmVersionFeatures & config.amd64BMI2) != 0) { features.add(AMD64.CPUFeature.BMI2); } - if ((config.x86CPUFeatures & config.cpuRTM) != 0) { + if ((config.vmVersionFeatures & config.amd64RTM) != 0) { features.add(AMD64.CPUFeature.RTM); } - if ((config.x86CPUFeatures & config.cpuADX) != 0) { + if ((config.vmVersionFeatures & config.amd64ADX) != 0) { features.add(AMD64.CPUFeature.ADX); } - if ((config.x86CPUFeatures & config.cpuAVX512F) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512F) != 0) { features.add(AMD64.CPUFeature.AVX512F); } - if ((config.x86CPUFeatures & config.cpuAVX512DQ) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512DQ) != 0) { features.add(AMD64.CPUFeature.AVX512DQ); } - if ((config.x86CPUFeatures & config.cpuAVX512PF) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512PF) != 0) { features.add(AMD64.CPUFeature.AVX512PF); } - if ((config.x86CPUFeatures & config.cpuAVX512ER) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512ER) != 0) { features.add(AMD64.CPUFeature.AVX512ER); } - if ((config.x86CPUFeatures & config.cpuAVX512CD) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512CD) != 0) { features.add(AMD64.CPUFeature.AVX512CD); } - if ((config.x86CPUFeatures & config.cpuAVX512BW) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512BW) != 0) { features.add(AMD64.CPUFeature.AVX512BW); } - if ((config.x86CPUFeatures & config.cpuAVX512VL) != 0) { + if ((config.vmVersionFeatures & config.amd64AVX512VL) != 0) { features.add(AMD64.CPUFeature.AVX512VL); } return features; diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.sparc/src/jdk/vm/ci/hotspot/sparc/SPARCHotSpotJVMCIBackendFactory.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.sparc/src/jdk/vm/ci/hotspot/sparc/SPARCHotSpotJVMCIBackendFactory.java index d9339c2fa7e..f0f0b05190e 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.sparc/src/jdk/vm/ci/hotspot/sparc/SPARCHotSpotJVMCIBackendFactory.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot.sparc/src/jdk/vm/ci/hotspot/sparc/SPARCHotSpotJVMCIBackendFactory.java @@ -60,73 +60,73 @@ public class SPARCHotSpotJVMCIBackendFactory implements HotSpotJVMCIBackendFacto protected EnumSet computeFeatures(HotSpotVMConfig config) { EnumSet features = EnumSet.noneOf(CPUFeature.class); - if ((config.sparcFeatures & config.vis1Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcVis1Instructions) != 0) { features.add(CPUFeature.VIS1); } - if ((config.sparcFeatures & config.vis2Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcVis2Instructions) != 0) { features.add(CPUFeature.VIS2); } - if ((config.sparcFeatures & config.vis3Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcVis3Instructions) != 0) { features.add(CPUFeature.VIS3); } - if ((config.sparcFeatures & config.cbcondInstructions) != 0) { + if ((config.vmVersionFeatures & config.sparcCbcondInstructions) != 0) { features.add(CPUFeature.CBCOND); } - if ((config.sparcFeatures & config.v8Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcV8Instructions) != 0) { features.add(CPUFeature.V8); } - if ((config.sparcFeatures & config.hardwareMul32) != 0) { + if ((config.vmVersionFeatures & config.sparcHardwareMul32) != 0) { features.add(CPUFeature.HARDWARE_MUL32); } - if ((config.sparcFeatures & config.hardwareDiv32) != 0) { + if ((config.vmVersionFeatures & config.sparcHardwareDiv32) != 0) { features.add(CPUFeature.HARDWARE_DIV32); } - if ((config.sparcFeatures & config.hardwareFsmuld) != 0) { + if ((config.vmVersionFeatures & config.sparcHardwareFsmuld) != 0) { features.add(CPUFeature.HARDWARE_FSMULD); } - if ((config.sparcFeatures & config.hardwarePopc) != 0) { + if ((config.vmVersionFeatures & config.sparcHardwarePopc) != 0) { features.add(CPUFeature.HARDWARE_POPC); } - if ((config.sparcFeatures & config.v9Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcV9Instructions) != 0) { features.add(CPUFeature.V9); } - if ((config.sparcFeatures & config.sun4v) != 0) { + if ((config.vmVersionFeatures & config.sparcSun4v) != 0) { features.add(CPUFeature.SUN4V); } - if ((config.sparcFeatures & config.blkInitInstructions) != 0) { + if ((config.vmVersionFeatures & config.sparcBlkInitInstructions) != 0) { features.add(CPUFeature.BLK_INIT_INSTRUCTIONS); } - if ((config.sparcFeatures & config.fmafInstructions) != 0) { + if ((config.vmVersionFeatures & config.sparcFmafInstructions) != 0) { features.add(CPUFeature.FMAF); } - if ((config.sparcFeatures & config.fmauInstructions) != 0) { + if ((config.vmVersionFeatures & config.sparcFmauInstructions) != 0) { features.add(CPUFeature.FMAU); } - if ((config.sparcFeatures & config.sparc64Family) != 0) { + if ((config.vmVersionFeatures & config.sparcSparc64Family) != 0) { features.add(CPUFeature.SPARC64_FAMILY); } - if ((config.sparcFeatures & config.mFamily) != 0) { + if ((config.vmVersionFeatures & config.sparcMFamily) != 0) { features.add(CPUFeature.M_FAMILY); } - if ((config.sparcFeatures & config.tFamily) != 0) { + if ((config.vmVersionFeatures & config.sparcTFamily) != 0) { features.add(CPUFeature.T_FAMILY); } - if ((config.sparcFeatures & config.t1Model) != 0) { + if ((config.vmVersionFeatures & config.sparcT1Model) != 0) { features.add(CPUFeature.T1_MODEL); } - if ((config.sparcFeatures & config.sparc5Instructions) != 0) { + if ((config.vmVersionFeatures & config.sparcSparc5Instructions) != 0) { features.add(CPUFeature.SPARC5); } - if ((config.sparcFeatures & config.aesInstructions) != 0) { + if ((config.vmVersionFeatures & config.sparcAesInstructions) != 0) { features.add(CPUFeature.SPARC64_FAMILY); } - if ((config.sparcFeatures & config.sha1Instruction) != 0) { + if ((config.vmVersionFeatures & config.sparcSha1Instruction) != 0) { features.add(CPUFeature.SHA1); } - if ((config.sparcFeatures & config.sha256Instruction) != 0) { + if ((config.vmVersionFeatures & config.sparcSha256Instruction) != 0) { features.add(CPUFeature.SHA256); } - if ((config.sparcFeatures & config.sha512Instruction) != 0) { + if ((config.vmVersionFeatures & config.sparcSha512Instruction) != 0) { features.add(CPUFeature.SHA512); } return features; diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantReflectionProvider.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantReflectionProvider.java index a5d165feb61..ffe53825947 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantReflectionProvider.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantReflectionProvider.java @@ -30,28 +30,22 @@ import java.lang.reflect.Array; import jdk.vm.ci.meta.Constant; import jdk.vm.ci.meta.ConstantReflectionProvider; import jdk.vm.ci.meta.JavaConstant; -import jdk.vm.ci.meta.JavaField; import jdk.vm.ci.meta.JavaKind; import jdk.vm.ci.meta.JavaType; import jdk.vm.ci.meta.MemoryAccessProvider; import jdk.vm.ci.meta.MethodHandleAccessProvider; +import jdk.vm.ci.meta.ResolvedJavaField; import jdk.vm.ci.meta.ResolvedJavaType; -import jdk.vm.ci.options.Option; -import jdk.vm.ci.options.OptionType; -import jdk.vm.ci.options.OptionValue; -import jdk.vm.ci.options.StableOptionValue; /** * HotSpot implementation of {@link ConstantReflectionProvider}. */ public class HotSpotConstantReflectionProvider implements ConstantReflectionProvider, HotSpotProxified { - static class Options { - //@formatter:off - @Option(help = "Constant fold final fields with default values.", type = OptionType.Debug) - public static final OptionValue TrustFinalDefaultFields = new OptionValue<>(true); - //@formatter:on - } + /** + * Determines whether to treat {@code final} fields with default values as constant. + */ + private static final boolean TrustFinalDefaultFields = HotSpotJVMCIRuntime.getBooleanProperty("TrustFinalDefaultFields", true); protected final HotSpotJVMCIRuntimeProvider runtime; protected final HotSpotMethodHandleAccessProvider methodHandleAccess; @@ -239,10 +233,10 @@ public class HotSpotConstantReflectionProvider implements ConstantReflectionProv /** * Determines if a static field is constant for the purpose of - * {@link #readConstantFieldValue(JavaField, JavaConstant)}. + * {@link #readConstantFieldValue(ResolvedJavaField, JavaConstant)}. */ protected boolean isStaticFieldConstant(HotSpotResolvedJavaField staticField) { - if (staticField.isFinal() || staticField.isStable()) { + if (staticField.isFinal() || (staticField.isStable() && runtime.getConfig().foldStableValues)) { ResolvedJavaType holder = staticField.getDeclaringClass(); if (holder.isInitialized() && !holder.getName().equals(SystemClassName)) { return true; @@ -255,14 +249,14 @@ public class HotSpotConstantReflectionProvider implements ConstantReflectionProv * Determines if a value read from a {@code final} instance field is considered constant. The * implementation in {@link HotSpotConstantReflectionProvider} returns true if {@code value} is * not the {@link JavaConstant#isDefaultForKind default value} for its kind or if - * {@link Options#TrustFinalDefaultFields} is true. + * {@link #TrustFinalDefaultFields} is true. * * @param value a value read from a {@code final} instance field * @param receiverClass the {@link Object#getClass() class} of object from which the * {@code value} was read */ protected boolean isFinalInstanceFieldValueConstant(JavaConstant value, Class receiverClass) { - return !value.isDefaultForKind() || Options.TrustFinalDefaultFields.getValue(); + return !value.isDefaultForKind() || TrustFinalDefaultFields; } /** @@ -278,13 +272,7 @@ public class HotSpotConstantReflectionProvider implements ConstantReflectionProv return !value.isDefaultForKind(); } - /** - * {@inheritDoc} - *

    - * The {@code value} field in {@link OptionValue} is considered constant if the type of - * {@code receiver} is (assignable to) {@link StableOptionValue}. - */ - public JavaConstant readConstantFieldValue(JavaField field, JavaConstant receiver) { + public JavaConstant readConstantFieldValue(ResolvedJavaField field, JavaConstant receiver) { HotSpotResolvedJavaField hotspotField = (HotSpotResolvedJavaField) field; if (hotspotField.isStatic()) { @@ -312,37 +300,31 @@ public class HotSpotConstantReflectionProvider implements ConstantReflectionProv return value; } } - } else if (hotspotField.isStable()) { + } else if (hotspotField.isStable() && runtime.getConfig().foldStableValues) { if (hotspotField.isInObject(object)) { JavaConstant value = readFieldValue(field, receiver); if (isStableInstanceFieldValueConstant(value, object.getClass())) { return value; } } - } else { - Class clazz = object.getClass(); - if (StableOptionValue.class.isAssignableFrom(clazz)) { - if (hotspotField.isInObject(object) && hotspotField.getName().equals("value")) { - StableOptionValue option = (StableOptionValue) object; - return HotSpotObjectConstantImpl.forObject(option.getValue()); - } - } } } } return null; } - public JavaConstant readFieldValue(JavaField field, JavaConstant receiver) { + public JavaConstant readFieldValue(ResolvedJavaField field, JavaConstant receiver) { HotSpotResolvedJavaField hotspotField = (HotSpotResolvedJavaField) field; if (!hotspotField.isStable()) { return readNonStableFieldValue(field, receiver); - } else { + } else if (runtime.getConfig().foldStableValues) { return readStableFieldValue(field, receiver, hotspotField.isDefaultStable()); + } else { + return null; } } - private JavaConstant readNonStableFieldValue(JavaField field, JavaConstant receiver) { + private JavaConstant readNonStableFieldValue(ResolvedJavaField field, JavaConstant receiver) { HotSpotResolvedJavaField hotspotField = (HotSpotResolvedJavaField) field; if (hotspotField.isStatic()) { HotSpotResolvedJavaType holder = (HotSpotResolvedJavaType) hotspotField.getDeclaringClass(); @@ -357,7 +339,7 @@ public class HotSpotConstantReflectionProvider implements ConstantReflectionProv return null; } - public JavaConstant readStableFieldValue(JavaField field, JavaConstant receiver, boolean isDefaultStable) { + public JavaConstant readStableFieldValue(ResolvedJavaField field, JavaConstant receiver, boolean isDefaultStable) { JavaConstant fieldValue = readNonStableFieldValue(field, receiver); if (fieldValue.isNonNull()) { JavaType declaredType = field.getType(); diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java index 7ea4ebf407d..bf7f255d4db 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java @@ -49,6 +49,7 @@ import jdk.vm.ci.runtime.JVMCI; import jdk.vm.ci.runtime.JVMCIBackend; import jdk.vm.ci.runtime.JVMCICompiler; import jdk.vm.ci.service.Services; +import sun.misc.VM; //JaCoCo Exclude @@ -83,6 +84,22 @@ public final class HotSpotJVMCIRuntime implements HotSpotJVMCIRuntimeProvider, H return DelayedInit.instance; } + /** + * Gets a boolean value based on a system property {@linkplain VM#getSavedProperty(String) + * saved} at system initialization time. The property name is prefixed with "{@code jvmci.}". + * + * @param name the name of the system property to derive a boolean value from using + * {@link Boolean#parseBoolean(String)} + * @param def the value to return if there is no system property corresponding to {@code name} + */ + public static boolean getBooleanProperty(String name, boolean def) { + String value = VM.getSavedProperty("jvmci." + name); + if (value == null) { + return def; + } + return Boolean.parseBoolean(value); + } + public static HotSpotJVMCIBackendFactory findFactory(String architecture) { for (HotSpotJVMCIBackendFactory factory : Services.load(HotSpotJVMCIBackendFactory.class)) { if (factory.getArchitecture().equalsIgnoreCase(architecture)) { diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaField.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaField.java index 00b0c0de8c9..1fd7e48924f 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaField.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaField.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2011, 2014, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011, 2015, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -40,17 +40,14 @@ public interface HotSpotResolvedJavaField extends ResolvedJavaField { int offset(); /** - * Checks if this field has the {@link Stable} annotation. - * - * @return true if field has {@link Stable} annotation, false otherwise + * Determines if this field should be treated as a constant. */ boolean isStable(); /** - * If this field is stable, checks if default values (0, null, etc.) should be considered stable - * as well. - * - * @return true if default values should be considered stable, false otherwise + * Determines if this field should be considered constant if it has the default value for its + * type (e.g, 0, null, etc.). The result of this method is undefined if this field is not + * {@linkplain #isStable() stable}. */ boolean isDefaultStable(); } diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaFieldImpl.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaFieldImpl.java index cd89be3e563..c6f4dabc0fb 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaFieldImpl.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaFieldImpl.java @@ -35,21 +35,16 @@ import jdk.vm.ci.meta.MetaAccessProvider; import jdk.vm.ci.meta.ModifiersProvider; import jdk.vm.ci.meta.ResolvedJavaField; import jdk.vm.ci.meta.ResolvedJavaType; -import jdk.vm.ci.options.Option; -import jdk.vm.ci.options.OptionType; -import jdk.vm.ci.options.OptionValue; /** * Represents a field in a HotSpot type. */ class HotSpotResolvedJavaFieldImpl implements HotSpotResolvedJavaField, HotSpotProxified { - static class Options { - //@formatter:off - @Option(help = "Mark well-known stable fields as such.", type = OptionType.Debug) - public static final OptionValue ImplicitStableValues = new OptionValue<>(true); - //@formatter:on - } + /** + * Mark well-known stable fields as such. + */ + private static final boolean ImplicitStableValues = HotSpotJVMCIRuntime.getBooleanProperty("ImplicitStableValues", true); private final HotSpotResolvedObjectTypeImpl holder; private final String name; @@ -190,7 +185,7 @@ class HotSpotResolvedJavaFieldImpl implements HotSpotResolvedJavaField, HotSpotP @Override public boolean isSynthetic() { - return (config().syntheticFlag & modifiers) != 0; + return (config().jvmAccSynthetic & modifiers) != 0; } /** @@ -203,7 +198,7 @@ class HotSpotResolvedJavaFieldImpl implements HotSpotResolvedJavaField, HotSpotP return true; } assert getAnnotation(Stable.class) == null; - if (Options.ImplicitStableValues.getValue() && isImplicitStableField()) { + if (ImplicitStableValues && isImplicitStableField()) { return true; } return false; diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethod.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethod.java index d1ecdac098d..80635caa7c7 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethod.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethod.java @@ -27,22 +27,12 @@ import java.lang.reflect.Modifier; import jdk.vm.ci.meta.JavaMethod; import jdk.vm.ci.meta.ResolvedJavaMethod; import jdk.vm.ci.meta.ResolvedJavaType; -import jdk.vm.ci.options.Option; -import jdk.vm.ci.options.OptionType; -import jdk.vm.ci.options.OptionValue; /** * Implementation of {@link JavaMethod} for resolved HotSpot methods. */ public interface HotSpotResolvedJavaMethod extends ResolvedJavaMethod { - public static class Options { - // @formatter:off - @Option(help = "", type = OptionType.Debug) - public static final OptionValue UseProfilingInformation = new OptionValue<>(true); - // @formatter:on - } - /** * Returns true if this method has a {@code CallerSensitive} annotation. * diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethodImpl.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethodImpl.java index 34647ae886c..5999d8c204f 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethodImpl.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedJavaMethodImpl.java @@ -24,7 +24,6 @@ package jdk.vm.ci.hotspot; import static jdk.vm.ci.hotspot.CompilerToVM.compilerToVM; import static jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime; -import static jdk.vm.ci.hotspot.HotSpotResolvedJavaMethod.Options.UseProfilingInformation; import static jdk.vm.ci.hotspot.HotSpotVMConfig.config; import static jdk.vm.ci.hotspot.UnsafeAccess.UNSAFE; @@ -424,7 +423,7 @@ final class HotSpotResolvedJavaMethodImpl extends HotSpotMethod implements HotSp public ProfilingInfo getProfilingInfo(boolean includeNormal, boolean includeOSR) { ProfilingInfo info; - if (UseProfilingInformation.getValue() && methodData == null) { + if (methodData == null) { long metaspaceMethodData = UNSAFE.getAddress(metaspaceMethod + config().methodDataOffset); if (metaspaceMethodData != 0) { methodData = new HotSpotMethodData(metaspaceMethodData, this); diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java index 4ef227ec823..65d92b1d8fd 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java @@ -324,8 +324,7 @@ final class HotSpotResolvedObjectTypeImpl extends HotSpotResolvedJavaType implem @Override public boolean hasFinalizer() { - HotSpotVMConfig config = config(); - return (getAccessFlags() & config.klassHasFinalizerFlag) != 0; + return (getAccessFlags() & config().jvmAccHasFinalizer) != 0; } @Override diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotVMConfig.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotVMConfig.java index 2f8decbdeab..d237006dfcb 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotVMConfig.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotVMConfig.java @@ -37,7 +37,6 @@ import jdk.vm.ci.hotspotvmconfig.HotSpotVMConstant; import jdk.vm.ci.hotspotvmconfig.HotSpotVMData; import jdk.vm.ci.hotspotvmconfig.HotSpotVMField; import jdk.vm.ci.hotspotvmconfig.HotSpotVMFlag; -import jdk.vm.ci.hotspotvmconfig.HotSpotVMManual; import jdk.vm.ci.hotspotvmconfig.HotSpotVMType; import sun.misc.Unsafe; @@ -68,11 +67,11 @@ public class HotSpotVMConfig { assert gHotSpotVMData != 0; // Make FindBugs happy. - gHotSpotVMStructs = 0; - gHotSpotVMTypes = 0; - gHotSpotVMIntConstants = 0; - gHotSpotVMLongConstants = 0; - gHotSpotVMAddresses = 0; + jvmciHotSpotVMStructs = 0; + jvmciHotSpotVMTypes = 0; + jvmciHotSpotVMIntConstants = 0; + jvmciHotSpotVMLongConstants = 0; + jvmciHotSpotVMAddresses = 0; // Initialize the gHotSpotVM fields. for (Field f : HotSpotVMConfig.class.getDeclaredFields()) { @@ -89,41 +88,17 @@ public class HotSpotVMConfig { } // Quick sanity check. - assert gHotSpotVMStructs != 0; - assert gHotSpotVMTypes != 0; - assert gHotSpotVMIntConstants != 0; - assert gHotSpotVMLongConstants != 0; - assert gHotSpotVMAddresses != 0; + assert jvmciHotSpotVMStructs != 0; + assert jvmciHotSpotVMTypes != 0; + assert jvmciHotSpotVMIntConstants != 0; + assert jvmciHotSpotVMLongConstants != 0; + assert jvmciHotSpotVMAddresses != 0; initialize(); oopEncoding = new CompressEncoding(narrowOopBase, narrowOopShift, logMinObjAlignment()); klassEncoding = new CompressEncoding(narrowKlassBase, narrowKlassShift, logKlassAlignment); - final long barrierSetAddress = UNSAFE.getAddress(universeCollectedHeap + collectedHeapBarrierSetOffset); - final int kind = UNSAFE.getInt(barrierSetAddress + barrierSetFakeRttiOffset + fakeRttiConcreteTagOffset); - if ((kind == barrierSetCardTableModRef) || (kind == barrierSetCardTableForRS) || (kind == barrierSetCardTableExtension) || (kind == barrierSetG1SATBCT) || (kind == barrierSetG1SATBCTLogging)) { - final long base = UNSAFE.getAddress(barrierSetAddress + cardTableModRefBSByteMapBaseOffset); - assert base != 0 : "unexpected byte_map_base: " + base; - cardtableStartAddress = base; - cardtableShift = cardTableModRefBSCardShift; - } else if (kind == barrierSetModRef) { - // No post barriers - cardtableStartAddress = 0; - cardtableShift = 0; - } else { - cardtableStartAddress = -1; - cardtableShift = -1; - } - - // Now handle all HotSpotVMManual fields. - inlineCacheMissStub = inlineCacheMissBlob + UNSAFE.getInt(inlineCacheMissBlob + codeBlobCodeOffsetOffset); - handleWrongMethodStub = wrongMethodBlob + UNSAFE.getInt(wrongMethodBlob + codeBlobCodeOffsetOffset); - handleDeoptStub = deoptBlob + UNSAFE.getInt(deoptBlob + codeBlobCodeOffsetOffset) + UNSAFE.getInt(deoptBlob + deoptimizationBlobUnpackOffsetOffset); - uncommonTrapStub = deoptBlob + UNSAFE.getInt(deoptBlob + codeBlobCodeOffsetOffset) + UNSAFE.getInt(deoptBlob + deoptimizationBlobUncommonTrapOffsetOffset); - - tlabAlignmentReserve = roundUp(threadLocalAllocBufferEndReserve(), minObjAlignment()); - assert check(); assert HotSpotVMConfigVerifier.check(); } @@ -139,28 +114,28 @@ public class HotSpotVMConfig { private void initialize() { // Fill the VM fields hash map. HashMap vmFields = new HashMap<>(); - for (VMFields.Field e : new VMFields(gHotSpotVMStructs)) { + for (VMFields.Field e : new VMFields(jvmciHotSpotVMStructs)) { vmFields.put(e.getName(), e); } // Fill the VM types hash map. HashMap vmTypes = new HashMap<>(); - for (VMTypes.Type e : new VMTypes(gHotSpotVMTypes)) { + for (VMTypes.Type e : new VMTypes(jvmciHotSpotVMTypes)) { vmTypes.put(e.getTypeName(), e); } // Fill the VM constants hash map. HashMap vmConstants = new HashMap<>(); - for (AbstractConstant e : new VMIntConstants(gHotSpotVMIntConstants)) { + for (AbstractConstant e : new VMIntConstants(jvmciHotSpotVMIntConstants)) { vmConstants.put(e.getName(), e); } - for (AbstractConstant e : new VMAddresses(gHotSpotVMLongConstants)) { + for (AbstractConstant e : new VMLongConstants(jvmciHotSpotVMLongConstants)) { vmConstants.put(e.getName(), e); } // Fill the VM addresses hash map. HashMap vmAddresses = new HashMap<>(); - for (VMAddresses.Address e : new VMAddresses(gHotSpotVMAddresses)) { + for (VMAddresses.Address e : new VMAddresses(jvmciHotSpotVMAddresses)) { vmAddresses.put(e.getName(), e); } @@ -213,6 +188,7 @@ public class HotSpotVMConfig { if (entry == null) { throw new JVMCIError(f.getName() + ": expected VM type not found: " + name); } + switch (annotation.get()) { case SIZE: setField(f, entry.getSize()); @@ -371,14 +347,14 @@ public class HotSpotVMConfig { /** * VMStructEntry (see {@code vmStructs.hpp}). */ - @HotSpotVMData(index = 0) @Stable private long gHotSpotVMStructs; - @HotSpotVMData(index = 1) @Stable private long gHotSpotVMStructEntryTypeNameOffset; - @HotSpotVMData(index = 2) @Stable private long gHotSpotVMStructEntryFieldNameOffset; - @HotSpotVMData(index = 3) @Stable private long gHotSpotVMStructEntryTypeStringOffset; - @HotSpotVMData(index = 4) @Stable private long gHotSpotVMStructEntryIsStaticOffset; - @HotSpotVMData(index = 5) @Stable private long gHotSpotVMStructEntryOffsetOffset; - @HotSpotVMData(index = 6) @Stable private long gHotSpotVMStructEntryAddressOffset; - @HotSpotVMData(index = 7) @Stable private long gHotSpotVMStructEntryArrayStride; + @HotSpotVMData(index = 0) @Stable private long jvmciHotSpotVMStructs; + @HotSpotVMData(index = 1) @Stable private long jvmciHotSpotVMStructEntryTypeNameOffset; + @HotSpotVMData(index = 2) @Stable private long jvmciHotSpotVMStructEntryFieldNameOffset; + @HotSpotVMData(index = 3) @Stable private long jvmciHotSpotVMStructEntryTypeStringOffset; + @HotSpotVMData(index = 4) @Stable private long jvmciHotSpotVMStructEntryIsStaticOffset; + @HotSpotVMData(index = 5) @Stable private long jvmciHotSpotVMStructEntryOffsetOffset; + @HotSpotVMData(index = 6) @Stable private long jvmciHotSpotVMStructEntryAddressOffset; + @HotSpotVMData(index = 7) @Stable private long jvmciHotSpotVMStructEntryArrayStride; final class VMFields implements Iterable { @@ -394,7 +370,7 @@ public class HotSpotVMConfig { private int index = 0; private Field current() { - return new Field(address + gHotSpotVMStructEntryArrayStride * index); + return new Field(address + jvmciHotSpotVMStructEntryArrayStride * index); } /** @@ -422,30 +398,30 @@ public class HotSpotVMConfig { } public String getTypeName() { - long typeNameAddress = UNSAFE.getAddress(entryAddress + gHotSpotVMStructEntryTypeNameOffset); + long typeNameAddress = UNSAFE.getAddress(entryAddress + jvmciHotSpotVMStructEntryTypeNameOffset); return readCString(UNSAFE, typeNameAddress); } public String getFieldName() { - long fieldNameAddress = UNSAFE.getAddress(entryAddress + gHotSpotVMStructEntryFieldNameOffset); + long fieldNameAddress = UNSAFE.getAddress(entryAddress + jvmciHotSpotVMStructEntryFieldNameOffset); return readCString(UNSAFE, fieldNameAddress); } public String getTypeString() { - long typeStringAddress = UNSAFE.getAddress(entryAddress + gHotSpotVMStructEntryTypeStringOffset); + long typeStringAddress = UNSAFE.getAddress(entryAddress + jvmciHotSpotVMStructEntryTypeStringOffset); return readCString(UNSAFE, typeStringAddress); } public boolean isStatic() { - return UNSAFE.getInt(entryAddress + gHotSpotVMStructEntryIsStaticOffset) != 0; + return UNSAFE.getInt(entryAddress + jvmciHotSpotVMStructEntryIsStaticOffset) != 0; } public long getOffset() { - return UNSAFE.getLong(entryAddress + gHotSpotVMStructEntryOffsetOffset); + return UNSAFE.getLong(entryAddress + jvmciHotSpotVMStructEntryOffsetOffset); } public long getAddress() { - return UNSAFE.getAddress(entryAddress + gHotSpotVMStructEntryAddressOffset); + return UNSAFE.getAddress(entryAddress + jvmciHotSpotVMStructEntryAddressOffset); } public String getName() { @@ -466,6 +442,7 @@ public class HotSpotVMConfig { case "address": case "intptr_t": case "uintptr_t": + case "size_t": return UNSAFE.getAddress(getAddress()); default: // All foo* types are addresses. @@ -487,14 +464,14 @@ public class HotSpotVMConfig { /** * VMTypeEntry (see vmStructs.hpp). */ - @HotSpotVMData(index = 8) @Stable private long gHotSpotVMTypes; - @HotSpotVMData(index = 9) @Stable private long gHotSpotVMTypeEntryTypeNameOffset; - @HotSpotVMData(index = 10) @Stable private long gHotSpotVMTypeEntrySuperclassNameOffset; - @HotSpotVMData(index = 11) @Stable private long gHotSpotVMTypeEntryIsOopTypeOffset; - @HotSpotVMData(index = 12) @Stable private long gHotSpotVMTypeEntryIsIntegerTypeOffset; - @HotSpotVMData(index = 13) @Stable private long gHotSpotVMTypeEntryIsUnsignedOffset; - @HotSpotVMData(index = 14) @Stable private long gHotSpotVMTypeEntrySizeOffset; - @HotSpotVMData(index = 15) @Stable private long gHotSpotVMTypeEntryArrayStride; + @HotSpotVMData(index = 8) @Stable private long jvmciHotSpotVMTypes; + @HotSpotVMData(index = 9) @Stable private long jvmciHotSpotVMTypeEntryTypeNameOffset; + @HotSpotVMData(index = 10) @Stable private long jvmciHotSpotVMTypeEntrySuperclassNameOffset; + @HotSpotVMData(index = 11) @Stable private long jvmciHotSpotVMTypeEntryIsOopTypeOffset; + @HotSpotVMData(index = 12) @Stable private long jvmciHotSpotVMTypeEntryIsIntegerTypeOffset; + @HotSpotVMData(index = 13) @Stable private long jvmciHotSpotVMTypeEntryIsUnsignedOffset; + @HotSpotVMData(index = 14) @Stable private long jvmciHotSpotVMTypeEntrySizeOffset; + @HotSpotVMData(index = 15) @Stable private long jvmciHotSpotVMTypeEntryArrayStride; final class VMTypes implements Iterable { @@ -510,7 +487,7 @@ public class HotSpotVMConfig { private int index = 0; private Type current() { - return new Type(address + gHotSpotVMTypeEntryArrayStride * index); + return new Type(address + jvmciHotSpotVMTypeEntryArrayStride * index); } /** @@ -538,29 +515,29 @@ public class HotSpotVMConfig { } public String getTypeName() { - long typeNameAddress = UNSAFE.getAddress(entryAddress + gHotSpotVMTypeEntryTypeNameOffset); + long typeNameAddress = UNSAFE.getAddress(entryAddress + jvmciHotSpotVMTypeEntryTypeNameOffset); return readCString(UNSAFE, typeNameAddress); } public String getSuperclassName() { - long superclassNameAddress = UNSAFE.getAddress(entryAddress + gHotSpotVMTypeEntrySuperclassNameOffset); + long superclassNameAddress = UNSAFE.getAddress(entryAddress + jvmciHotSpotVMTypeEntrySuperclassNameOffset); return readCString(UNSAFE, superclassNameAddress); } public boolean isOopType() { - return UNSAFE.getInt(entryAddress + gHotSpotVMTypeEntryIsOopTypeOffset) != 0; + return UNSAFE.getInt(entryAddress + jvmciHotSpotVMTypeEntryIsOopTypeOffset) != 0; } public boolean isIntegerType() { - return UNSAFE.getInt(entryAddress + gHotSpotVMTypeEntryIsIntegerTypeOffset) != 0; + return UNSAFE.getInt(entryAddress + jvmciHotSpotVMTypeEntryIsIntegerTypeOffset) != 0; } public boolean isUnsigned() { - return UNSAFE.getInt(entryAddress + gHotSpotVMTypeEntryIsUnsignedOffset) != 0; + return UNSAFE.getInt(entryAddress + jvmciHotSpotVMTypeEntryIsUnsignedOffset) != 0; } public long getSize() { - return UNSAFE.getLong(entryAddress + gHotSpotVMTypeEntrySizeOffset); + return UNSAFE.getLong(entryAddress + jvmciHotSpotVMTypeEntrySizeOffset); } @Override @@ -594,10 +571,10 @@ public class HotSpotVMConfig { /** * VMIntConstantEntry (see vmStructs.hpp). */ - @HotSpotVMData(index = 16) @Stable private long gHotSpotVMIntConstants; - @HotSpotVMData(index = 17) @Stable private long gHotSpotVMIntConstantEntryNameOffset; - @HotSpotVMData(index = 18) @Stable private long gHotSpotVMIntConstantEntryValueOffset; - @HotSpotVMData(index = 19) @Stable private long gHotSpotVMIntConstantEntryArrayStride; + @HotSpotVMData(index = 16) @Stable private long jvmciHotSpotVMIntConstants; + @HotSpotVMData(index = 17) @Stable private long jvmciHotSpotVMIntConstantEntryNameOffset; + @HotSpotVMData(index = 18) @Stable private long jvmciHotSpotVMIntConstantEntryValueOffset; + @HotSpotVMData(index = 19) @Stable private long jvmciHotSpotVMIntConstantEntryArrayStride; final class VMIntConstants implements Iterable { @@ -613,7 +590,7 @@ public class HotSpotVMConfig { private int index = 0; private Constant current() { - return new Constant(address + gHotSpotVMIntConstantEntryArrayStride * index); + return new Constant(address + jvmciHotSpotVMIntConstantEntryArrayStride * index); } /** @@ -635,7 +612,7 @@ public class HotSpotVMConfig { final class Constant extends AbstractConstant { Constant(long address) { - super(address, gHotSpotVMIntConstantEntryNameOffset, gHotSpotVMIntConstantEntryValueOffset); + super(address, jvmciHotSpotVMIntConstantEntryNameOffset, jvmciHotSpotVMIntConstantEntryValueOffset); } @Override @@ -653,10 +630,10 @@ public class HotSpotVMConfig { /** * VMLongConstantEntry (see vmStructs.hpp). */ - @HotSpotVMData(index = 20) @Stable private long gHotSpotVMLongConstants; - @HotSpotVMData(index = 21) @Stable private long gHotSpotVMLongConstantEntryNameOffset; - @HotSpotVMData(index = 22) @Stable private long gHotSpotVMLongConstantEntryValueOffset; - @HotSpotVMData(index = 23) @Stable private long gHotSpotVMLongConstantEntryArrayStride; + @HotSpotVMData(index = 20) @Stable private long jvmciHotSpotVMLongConstants; + @HotSpotVMData(index = 21) @Stable private long jvmciHotSpotVMLongConstantEntryNameOffset; + @HotSpotVMData(index = 22) @Stable private long jvmciHotSpotVMLongConstantEntryValueOffset; + @HotSpotVMData(index = 23) @Stable private long jvmciHotSpotVMLongConstantEntryArrayStride; final class VMLongConstants implements Iterable { @@ -672,7 +649,7 @@ public class HotSpotVMConfig { private int index = 0; private Constant currentEntry() { - return new Constant(address + gHotSpotVMLongConstantEntryArrayStride * index); + return new Constant(address + jvmciHotSpotVMLongConstantEntryArrayStride * index); } /** @@ -694,7 +671,7 @@ public class HotSpotVMConfig { final class Constant extends AbstractConstant { Constant(long address) { - super(address, gHotSpotVMLongConstantEntryNameOffset, gHotSpotVMLongConstantEntryValueOffset); + super(address, jvmciHotSpotVMLongConstantEntryNameOffset, jvmciHotSpotVMLongConstantEntryValueOffset); } @Override @@ -712,10 +689,10 @@ public class HotSpotVMConfig { /** * VMAddressEntry (see vmStructs.hpp). */ - @HotSpotVMData(index = 24) @Stable private long gHotSpotVMAddresses; - @HotSpotVMData(index = 25) @Stable private long gHotSpotVMAddressEntryNameOffset; - @HotSpotVMData(index = 26) @Stable private long gHotSpotVMAddressEntryValueOffset; - @HotSpotVMData(index = 27) @Stable private long gHotSpotVMAddressEntryArrayStride; + @HotSpotVMData(index = 24) @Stable private long jvmciHotSpotVMAddresses; + @HotSpotVMData(index = 25) @Stable private long jvmciHotSpotVMAddressEntryNameOffset; + @HotSpotVMData(index = 26) @Stable private long jvmciHotSpotVMAddressEntryValueOffset; + @HotSpotVMData(index = 27) @Stable private long jvmciHotSpotVMAddressEntryArrayStride; final class VMAddresses implements Iterable { @@ -731,7 +708,7 @@ public class HotSpotVMConfig { private int index = 0; private Address currentEntry() { - return new Address(address + gHotSpotVMAddressEntryArrayStride * index); + return new Address(address + jvmciHotSpotVMAddressEntryArrayStride * index); } /** @@ -753,7 +730,7 @@ public class HotSpotVMConfig { final class Address extends AbstractConstant { Address(long address) { - super(address, gHotSpotVMAddressEntryNameOffset, gHotSpotVMAddressEntryValueOffset); + super(address, jvmciHotSpotVMAddressEntryNameOffset, jvmciHotSpotVMAddressEntryValueOffset); } @Override @@ -876,6 +853,7 @@ public class HotSpotVMConfig { @HotSpotVMFlag(name = "JVMCIUseFastLocking") @Stable public boolean useFastLocking; @HotSpotVMFlag(name = "ForceUnreachable") @Stable public boolean forceUnreachable; @HotSpotVMFlag(name = "CodeCacheSegmentSize") @Stable public int codeSegmentSize; + @HotSpotVMFlag(name = "FoldStableValues") @Stable public boolean foldStableValues; @HotSpotVMFlag(name = "UseTLAB") @Stable public boolean useTLAB; @HotSpotVMFlag(name = "UseBiasedLocking") @Stable public boolean useBiasedLocking; @@ -896,7 +874,7 @@ public class HotSpotVMConfig { @HotSpotVMFlag(name = "FlightRecorder", optional = true) @Stable public boolean flightRecorder; - @HotSpotVMField(name = "Universe::_collectedHeap", type = "CollectedHeap*", get = HotSpotVMField.Type.VALUE) @Stable private long universeCollectedHeap; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_collectedHeap", type = "CollectedHeap*", get = HotSpotVMField.Type.VALUE) @Stable private long universeCollectedHeap; @HotSpotVMField(name = "CollectedHeap::_total_collections", type = "unsigned int", get = HotSpotVMField.Type.OFFSET) @Stable private int collectedHeapTotalCollectionsOffset; public long gcTotalCollectionsAddress() { @@ -909,8 +887,8 @@ public class HotSpotVMConfig { @HotSpotVMFlag(name = "UseCompressedOops") @Stable public boolean useCompressedOops; @HotSpotVMFlag(name = "UseCompressedClassPointers") @Stable public boolean useCompressedClassPointers; - @HotSpotVMField(name = "Universe::_narrow_oop._base", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long narrowOopBase; - @HotSpotVMField(name = "Universe::_narrow_oop._shift", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int narrowOopShift; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_narrow_oop_base", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long narrowOopBase; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_narrow_oop_shift", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int narrowOopShift; @HotSpotVMFlag(name = "ObjectAlignmentInBytes") @Stable public int objectAlignment; public final int minObjAlignment() { @@ -922,77 +900,75 @@ public class HotSpotVMConfig { } @HotSpotVMType(name = "narrowKlass", get = HotSpotVMType.Type.SIZE) @Stable public int narrowKlassSize; - @HotSpotVMField(name = "Universe::_narrow_klass._base", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long narrowKlassBase; - @HotSpotVMField(name = "Universe::_narrow_klass._shift", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int narrowKlassShift; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_narrow_klass_base", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long narrowKlassBase; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_narrow_klass_shift", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int narrowKlassShift; @HotSpotVMConstant(name = "LogKlassAlignmentInBytes") @Stable public int logKlassAlignment; // CPU capabilities @HotSpotVMFlag(name = "UseSSE") @Stable public int useSSE; @HotSpotVMFlag(name = "UseAVX", archs = {"amd64"}) @Stable public int useAVX; - @HotSpotVMField(name = "Abstract_VM_Version::_reserve_for_allocation_prefetch", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int abstractVmVersionReserveForAllocationPrefetch; + @HotSpotVMField(name = "Abstract_VM_Version::_features", type = "uint64_t", get = HotSpotVMField.Type.VALUE) @Stable public long vmVersionFeatures; - // X86 specific values - @HotSpotVMField(name = "VM_Version::_cpuFeatures", type = "uint64_t", get = HotSpotVMField.Type.VALUE, archs = {"amd64"}) @Stable public long x86CPUFeatures; - @HotSpotVMConstant(name = "VM_Version::CPU_CX8", archs = {"amd64"}) @Stable public long cpuCX8; - @HotSpotVMConstant(name = "VM_Version::CPU_CMOV", archs = {"amd64"}) @Stable public long cpuCMOV; - @HotSpotVMConstant(name = "VM_Version::CPU_FXSR", archs = {"amd64"}) @Stable public long cpuFXSR; - @HotSpotVMConstant(name = "VM_Version::CPU_HT", archs = {"amd64"}) @Stable public long cpuHT; - @HotSpotVMConstant(name = "VM_Version::CPU_MMX", archs = {"amd64"}) @Stable public long cpuMMX; - @HotSpotVMConstant(name = "VM_Version::CPU_3DNOW_PREFETCH", archs = {"amd64"}) @Stable public long cpu3DNOWPREFETCH; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE", archs = {"amd64"}) @Stable public long cpuSSE; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE2", archs = {"amd64"}) @Stable public long cpuSSE2; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE3", archs = {"amd64"}) @Stable public long cpuSSE3; - @HotSpotVMConstant(name = "VM_Version::CPU_SSSE3", archs = {"amd64"}) @Stable public long cpuSSSE3; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE4A", archs = {"amd64"}) @Stable public long cpuSSE4A; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE4_1", archs = {"amd64"}) @Stable public long cpuSSE41; - @HotSpotVMConstant(name = "VM_Version::CPU_SSE4_2", archs = {"amd64"}) @Stable public long cpuSSE42; - @HotSpotVMConstant(name = "VM_Version::CPU_POPCNT", archs = {"amd64"}) @Stable public long cpuPOPCNT; - @HotSpotVMConstant(name = "VM_Version::CPU_LZCNT", archs = {"amd64"}) @Stable public long cpuLZCNT; - @HotSpotVMConstant(name = "VM_Version::CPU_TSC", archs = {"amd64"}) @Stable public long cpuTSC; - @HotSpotVMConstant(name = "VM_Version::CPU_TSCINV", archs = {"amd64"}) @Stable public long cpuTSCINV; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX", archs = {"amd64"}) @Stable public long cpuAVX; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX2", archs = {"amd64"}) @Stable public long cpuAVX2; - @HotSpotVMConstant(name = "VM_Version::CPU_AES", archs = {"amd64"}) @Stable public long cpuAES; - @HotSpotVMConstant(name = "VM_Version::CPU_ERMS", archs = {"amd64"}) @Stable public long cpuERMS; - @HotSpotVMConstant(name = "VM_Version::CPU_CLMUL", archs = {"amd64"}) @Stable public long cpuCLMUL; - @HotSpotVMConstant(name = "VM_Version::CPU_BMI1", archs = {"amd64"}) @Stable public long cpuBMI1; - @HotSpotVMConstant(name = "VM_Version::CPU_BMI2", archs = {"amd64"}) @Stable public long cpuBMI2; - @HotSpotVMConstant(name = "VM_Version::CPU_RTM", archs = {"amd64"}) @Stable public long cpuRTM; - @HotSpotVMConstant(name = "VM_Version::CPU_ADX", archs = {"amd64"}) @Stable public long cpuADX; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512F", archs = {"amd64"}) @Stable public long cpuAVX512F; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512DQ", archs = {"amd64"}) @Stable public long cpuAVX512DQ; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512PF", archs = {"amd64"}) @Stable public long cpuAVX512PF; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512ER", archs = {"amd64"}) @Stable public long cpuAVX512ER; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512CD", archs = {"amd64"}) @Stable public long cpuAVX512CD; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512BW", archs = {"amd64"}) @Stable public long cpuAVX512BW; - @HotSpotVMConstant(name = "VM_Version::CPU_AVX512VL", archs = {"amd64"}) @Stable public long cpuAVX512VL; + // AMD64 specific values + @HotSpotVMConstant(name = "VM_Version::CPU_CX8", archs = {"amd64"}) @Stable public long amd64CX8; + @HotSpotVMConstant(name = "VM_Version::CPU_CMOV", archs = {"amd64"}) @Stable public long amd64CMOV; + @HotSpotVMConstant(name = "VM_Version::CPU_FXSR", archs = {"amd64"}) @Stable public long amd64FXSR; + @HotSpotVMConstant(name = "VM_Version::CPU_HT", archs = {"amd64"}) @Stable public long amd64HT; + @HotSpotVMConstant(name = "VM_Version::CPU_MMX", archs = {"amd64"}) @Stable public long amd64MMX; + @HotSpotVMConstant(name = "VM_Version::CPU_3DNOW_PREFETCH", archs = {"amd64"}) @Stable public long amd643DNOWPREFETCH; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE", archs = {"amd64"}) @Stable public long amd64SSE; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE2", archs = {"amd64"}) @Stable public long amd64SSE2; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE3", archs = {"amd64"}) @Stable public long amd64SSE3; + @HotSpotVMConstant(name = "VM_Version::CPU_SSSE3", archs = {"amd64"}) @Stable public long amd64SSSE3; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE4A", archs = {"amd64"}) @Stable public long amd64SSE4A; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE4_1", archs = {"amd64"}) @Stable public long amd64SSE41; + @HotSpotVMConstant(name = "VM_Version::CPU_SSE4_2", archs = {"amd64"}) @Stable public long amd64SSE42; + @HotSpotVMConstant(name = "VM_Version::CPU_POPCNT", archs = {"amd64"}) @Stable public long amd64POPCNT; + @HotSpotVMConstant(name = "VM_Version::CPU_LZCNT", archs = {"amd64"}) @Stable public long amd64LZCNT; + @HotSpotVMConstant(name = "VM_Version::CPU_TSC", archs = {"amd64"}) @Stable public long amd64TSC; + @HotSpotVMConstant(name = "VM_Version::CPU_TSCINV", archs = {"amd64"}) @Stable public long amd64TSCINV; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX", archs = {"amd64"}) @Stable public long amd64AVX; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX2", archs = {"amd64"}) @Stable public long amd64AVX2; + @HotSpotVMConstant(name = "VM_Version::CPU_AES", archs = {"amd64"}) @Stable public long amd64AES; + @HotSpotVMConstant(name = "VM_Version::CPU_ERMS", archs = {"amd64"}) @Stable public long amd64ERMS; + @HotSpotVMConstant(name = "VM_Version::CPU_CLMUL", archs = {"amd64"}) @Stable public long amd64CLMUL; + @HotSpotVMConstant(name = "VM_Version::CPU_BMI1", archs = {"amd64"}) @Stable public long amd64BMI1; + @HotSpotVMConstant(name = "VM_Version::CPU_BMI2", archs = {"amd64"}) @Stable public long amd64BMI2; + @HotSpotVMConstant(name = "VM_Version::CPU_RTM", archs = {"amd64"}) @Stable public long amd64RTM; + @HotSpotVMConstant(name = "VM_Version::CPU_ADX", archs = {"amd64"}) @Stable public long amd64ADX; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512F", archs = {"amd64"}) @Stable public long amd64AVX512F; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512DQ", archs = {"amd64"}) @Stable public long amd64AVX512DQ; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512PF", archs = {"amd64"}) @Stable public long amd64AVX512PF; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512ER", archs = {"amd64"}) @Stable public long amd64AVX512ER; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512CD", archs = {"amd64"}) @Stable public long amd64AVX512CD; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512BW", archs = {"amd64"}) @Stable public long amd64AVX512BW; + @HotSpotVMConstant(name = "VM_Version::CPU_AVX512VL", archs = {"amd64"}) @Stable public long amd64AVX512VL; // SPARC specific values - @HotSpotVMField(name = "VM_Version::_features", type = "int", get = HotSpotVMField.Type.VALUE, archs = {"sparc"}) @Stable public int sparcFeatures; - @HotSpotVMConstant(name = "VM_Version::vis3_instructions_m", archs = {"sparc"}) @Stable public int vis3Instructions; - @HotSpotVMConstant(name = "VM_Version::vis2_instructions_m", archs = {"sparc"}) @Stable public int vis2Instructions; - @HotSpotVMConstant(name = "VM_Version::vis1_instructions_m", archs = {"sparc"}) @Stable public int vis1Instructions; - @HotSpotVMConstant(name = "VM_Version::cbcond_instructions_m", archs = {"sparc"}) @Stable public int cbcondInstructions; - @HotSpotVMConstant(name = "VM_Version::v8_instructions_m", archs = {"sparc"}) @Stable public int v8Instructions; - @HotSpotVMConstant(name = "VM_Version::hardware_mul32_m", archs = {"sparc"}) @Stable public int hardwareMul32; - @HotSpotVMConstant(name = "VM_Version::hardware_div32_m", archs = {"sparc"}) @Stable public int hardwareDiv32; - @HotSpotVMConstant(name = "VM_Version::hardware_fsmuld_m", archs = {"sparc"}) @Stable public int hardwareFsmuld; - @HotSpotVMConstant(name = "VM_Version::hardware_popc_m", archs = {"sparc"}) @Stable public int hardwarePopc; - @HotSpotVMConstant(name = "VM_Version::v9_instructions_m", archs = {"sparc"}) @Stable public int v9Instructions; - @HotSpotVMConstant(name = "VM_Version::sun4v_m", archs = {"sparc"}) @Stable public int sun4v; - @HotSpotVMConstant(name = "VM_Version::blk_init_instructions_m", archs = {"sparc"}) @Stable public int blkInitInstructions; - @HotSpotVMConstant(name = "VM_Version::fmaf_instructions_m", archs = {"sparc"}) @Stable public int fmafInstructions; - @HotSpotVMConstant(name = "VM_Version::fmau_instructions_m", archs = {"sparc"}) @Stable public int fmauInstructions; - @HotSpotVMConstant(name = "VM_Version::sparc64_family_m", archs = {"sparc"}) @Stable public int sparc64Family; - @HotSpotVMConstant(name = "VM_Version::M_family_m", archs = {"sparc"}) @Stable public int mFamily; - @HotSpotVMConstant(name = "VM_Version::T_family_m", archs = {"sparc"}) @Stable public int tFamily; - @HotSpotVMConstant(name = "VM_Version::T1_model_m", archs = {"sparc"}) @Stable public int t1Model; - @HotSpotVMConstant(name = "VM_Version::sparc5_instructions_m", archs = {"sparc"}) @Stable public int sparc5Instructions; - @HotSpotVMConstant(name = "VM_Version::aes_instructions_m", archs = {"sparc"}) @Stable public int aesInstructions; - @HotSpotVMConstant(name = "VM_Version::sha1_instruction_m", archs = {"sparc"}) @Stable public int sha1Instruction; - @HotSpotVMConstant(name = "VM_Version::sha256_instruction_m", archs = {"sparc"}) @Stable public int sha256Instruction; - @HotSpotVMConstant(name = "VM_Version::sha512_instruction_m", archs = {"sparc"}) @Stable public int sha512Instruction; + @HotSpotVMConstant(name = "VM_Version::vis3_instructions_m", archs = {"sparc"}) @Stable public int sparcVis3Instructions; + @HotSpotVMConstant(name = "VM_Version::vis2_instructions_m", archs = {"sparc"}) @Stable public int sparcVis2Instructions; + @HotSpotVMConstant(name = "VM_Version::vis1_instructions_m", archs = {"sparc"}) @Stable public int sparcVis1Instructions; + @HotSpotVMConstant(name = "VM_Version::cbcond_instructions_m", archs = {"sparc"}) @Stable public int sparcCbcondInstructions; + @HotSpotVMConstant(name = "VM_Version::v8_instructions_m", archs = {"sparc"}) @Stable public int sparcV8Instructions; + @HotSpotVMConstant(name = "VM_Version::hardware_mul32_m", archs = {"sparc"}) @Stable public int sparcHardwareMul32; + @HotSpotVMConstant(name = "VM_Version::hardware_div32_m", archs = {"sparc"}) @Stable public int sparcHardwareDiv32; + @HotSpotVMConstant(name = "VM_Version::hardware_fsmuld_m", archs = {"sparc"}) @Stable public int sparcHardwareFsmuld; + @HotSpotVMConstant(name = "VM_Version::hardware_popc_m", archs = {"sparc"}) @Stable public int sparcHardwarePopc; + @HotSpotVMConstant(name = "VM_Version::v9_instructions_m", archs = {"sparc"}) @Stable public int sparcV9Instructions; + @HotSpotVMConstant(name = "VM_Version::sun4v_m", archs = {"sparc"}) @Stable public int sparcSun4v; + @HotSpotVMConstant(name = "VM_Version::blk_init_instructions_m", archs = {"sparc"}) @Stable public int sparcBlkInitInstructions; + @HotSpotVMConstant(name = "VM_Version::fmaf_instructions_m", archs = {"sparc"}) @Stable public int sparcFmafInstructions; + @HotSpotVMConstant(name = "VM_Version::fmau_instructions_m", archs = {"sparc"}) @Stable public int sparcFmauInstructions; + @HotSpotVMConstant(name = "VM_Version::sparc64_family_m", archs = {"sparc"}) @Stable public int sparcSparc64Family; + @HotSpotVMConstant(name = "VM_Version::M_family_m", archs = {"sparc"}) @Stable public int sparcMFamily; + @HotSpotVMConstant(name = "VM_Version::T_family_m", archs = {"sparc"}) @Stable public int sparcTFamily; + @HotSpotVMConstant(name = "VM_Version::T1_model_m", archs = {"sparc"}) @Stable public int sparcT1Model; + @HotSpotVMConstant(name = "VM_Version::sparc5_instructions_m", archs = {"sparc"}) @Stable public int sparcSparc5Instructions; + @HotSpotVMConstant(name = "VM_Version::aes_instructions_m", archs = {"sparc"}) @Stable public int sparcAesInstructions; + @HotSpotVMConstant(name = "VM_Version::sha1_instruction_m", archs = {"sparc"}) @Stable public int sparcSha1Instruction; + @HotSpotVMConstant(name = "VM_Version::sha256_instruction_m", archs = {"sparc"}) @Stable public int sparcSha256Instruction; + @HotSpotVMConstant(name = "VM_Version::sha512_instruction_m", archs = {"sparc"}) @Stable public int sparcSha512Instruction; @HotSpotVMFlag(name = "UseBlockZeroing", archs = {"sparc"}) @Stable public boolean useBlockZeroing; @HotSpotVMFlag(name = "BlockZeroingLowLimit", archs = {"sparc"}) @Stable public int blockZeroingLowLimit; @@ -1054,7 +1030,8 @@ public class HotSpotVMConfig { @HotSpotVMField(name = "InstanceKlass::_init_state", type = "u1", get = HotSpotVMField.Type.OFFSET) @Stable public int instanceKlassInitStateOffset; @HotSpotVMField(name = "InstanceKlass::_constants", type = "ConstantPool*", get = HotSpotVMField.Type.OFFSET) @Stable public int instanceKlassConstantsOffset; @HotSpotVMField(name = "InstanceKlass::_fields", type = "Array*", get = HotSpotVMField.Type.OFFSET) @Stable public int instanceKlassFieldsOffset; - @HotSpotVMField(name = "InstanceKlass::_vtable_len", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int instanceKlassVtableLengthOffset; + @HotSpotVMField(name = "CompilerToVM::Data::InstanceKlass_vtable_start_offset", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int instanceKlassVtableStartOffset; + @HotSpotVMField(name = "CompilerToVM::Data::InstanceKlass_vtable_length_offset", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int instanceKlassVtableLengthOffset; @HotSpotVMConstant(name = "InstanceKlass::linked") @Stable public int instanceKlassStateLinked; @HotSpotVMConstant(name = "InstanceKlass::fully_initialized") @Stable public int instanceKlassStateFullyInitialized; @@ -1063,12 +1040,7 @@ public class HotSpotVMConfig { * See {@code InstanceKlass::vtable_start_offset()}. */ public final int instanceKlassVtableStartOffset() { - return roundUp(instanceKlassSize, heapWordSize); - } - - // TODO use CodeUtil method once it's moved from NumUtil - private static int roundUp(int number, int mod) { - return ((number + mod - 1) / mod) * mod; + return instanceKlassVtableStartOffset * heapWordSize; } @HotSpotVMType(name = "arrayOopDesc", get = HotSpotVMType.Type.SIZE) @Stable public int arrayOopDescSize; @@ -1100,11 +1072,22 @@ public class HotSpotVMConfig { @HotSpotVMConstant(name = "FIELDINFO_TAG_SIZE") @Stable public int fieldInfoTagSize; + @HotSpotVMConstant(name = "JVM_ACC_MONITOR_MATCH") @Stable public int jvmAccMonitorMatch; + @HotSpotVMConstant(name = "JVM_ACC_HAS_MONITOR_BYTECODES") @Stable public int jvmAccHasMonitorBytecodes; + @HotSpotVMConstant(name = "JVM_ACC_HAS_FINALIZER") @Stable public int jvmAccHasFinalizer; @HotSpotVMConstant(name = "JVM_ACC_FIELD_INTERNAL") @Stable public int jvmAccFieldInternal; @HotSpotVMConstant(name = "JVM_ACC_FIELD_STABLE") @Stable public int jvmAccFieldStable; @HotSpotVMConstant(name = "JVM_ACC_FIELD_HAS_GENERIC_SIGNATURE") @Stable public int jvmAccFieldHasGenericSignature; @HotSpotVMConstant(name = "JVM_ACC_WRITTEN_FLAGS") @Stable public int jvmAccWrittenFlags; + // Modifier.SYNTHETIC is not public so we get it via vmStructs. + @HotSpotVMConstant(name = "JVM_ACC_SYNTHETIC") @Stable public int jvmAccSynthetic; + + /** + * @see HotSpotResolvedObjectTypeImpl#createField + */ + @HotSpotVMConstant(name = "JVM_RECOGNIZED_FIELD_MODIFIERS") @Stable public int recognizedFieldModifiers; + @HotSpotVMField(name = "Thread::_tlab", type = "ThreadLocalAllocBuffer", get = HotSpotVMField.Type.OFFSET) @Stable public int threadTlabOffset; @HotSpotVMField(name = "JavaThread::_anchor", type = "JavaFrameAnchor", get = HotSpotVMField.Type.OFFSET) @Stable public int javaThreadAnchorOffset; @@ -1202,16 +1185,17 @@ public class HotSpotVMConfig { @HotSpotVMField(name = "OSThread::_interrupted", type = "jint", get = HotSpotVMField.Type.OFFSET) @Stable public int osThreadInterruptedOffset; - @HotSpotVMConstant(name = "markOopDesc::unlocked_value") @Stable public int unlockedMask; + @HotSpotVMConstant(name = "markOopDesc::hash_shift") @Stable public long markOopDescHashShift; + @HotSpotVMConstant(name = "markOopDesc::biased_lock_mask_in_place") @Stable public int biasedLockMaskInPlace; @HotSpotVMConstant(name = "markOopDesc::age_mask_in_place") @Stable public int ageMaskInPlace; @HotSpotVMConstant(name = "markOopDesc::epoch_mask_in_place") @Stable public int epochMaskInPlace; - - @HotSpotVMConstant(name = "markOopDesc::hash_shift") @Stable public long markOopDescHashShift; @HotSpotVMConstant(name = "markOopDesc::hash_mask") @Stable public long markOopDescHashMask; @HotSpotVMConstant(name = "markOopDesc::hash_mask_in_place") @Stable public long markOopDescHashMaskInPlace; + @HotSpotVMConstant(name = "markOopDesc::unlocked_value") @Stable public int unlockedMask; @HotSpotVMConstant(name = "markOopDesc::biased_lock_pattern") @Stable public int biasedLockPattern; + @HotSpotVMConstant(name = "markOopDesc::no_hash_in_place") @Stable public int markWordNoHashInPlace; @HotSpotVMConstant(name = "markOopDesc::no_lock_in_place") @Stable public int markWordNoLockInPlace; @@ -1247,6 +1231,11 @@ public class HotSpotVMConfig { @HotSpotVMField(name = "Method::_flags", type = "u2", get = HotSpotVMField.Type.OFFSET) @Stable public int methodFlagsOffset; @HotSpotVMField(name = "Method::_vtable_index", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodVtableIndexOffset; + @HotSpotVMField(name = "Method::_method_counters", type = "MethodCounters*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCountersOffset; + @HotSpotVMField(name = "Method::_method_data", type = "MethodData*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOffset; + @HotSpotVMField(name = "Method::_from_compiled_entry", type = "address", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCompiledEntryOffset; + @HotSpotVMField(name = "Method::_code", type = "nmethod*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCodeOffset; + @HotSpotVMConstant(name = "Method::_jfr_towrite") @Stable public int methodFlagsJfrTowrite; @HotSpotVMConstant(name = "Method::_caller_sensitive") @Stable public int methodFlagsCallerSensitive; @HotSpotVMConstant(name = "Method::_force_inline") @Stable public int methodFlagsForceInline; @@ -1255,16 +1244,29 @@ public class HotSpotVMConfig { @HotSpotVMConstant(name = "Method::nonvirtual_vtable_index") @Stable public int nonvirtualVtableIndex; @HotSpotVMConstant(name = "Method::invalid_vtable_index") @Stable public int invalidVtableIndex; + @HotSpotVMField(name = "MethodCounters::_invocation_counter", type = "InvocationCounter", get = HotSpotVMField.Type.OFFSET) @Stable public int invocationCounterOffset; + @HotSpotVMField(name = "MethodCounters::_backedge_counter", type = "InvocationCounter", get = HotSpotVMField.Type.OFFSET) @Stable public int backedgeCounterOffset; + @HotSpotVMConstant(name = "InvocationCounter::count_increment") @Stable public int invocationCounterIncrement; + @HotSpotVMConstant(name = "InvocationCounter::count_shift") @Stable public int invocationCounterShift; + + @HotSpotVMField(name = "MethodData::_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataSize; + @HotSpotVMField(name = "MethodData::_data_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataDataSize; + @HotSpotVMField(name = "MethodData::_data[0]", type = "intptr_t", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOopDataOffset; + @HotSpotVMField(name = "MethodData::_trap_hist._array[0]", type = "u1", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOopTrapHistoryOffset; + @HotSpotVMField(name = "MethodData::_jvmci_ir_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataIRSizeOffset; + + @HotSpotVMField(name = "nmethod::_verified_entry_point", type = "address", get = HotSpotVMField.Type.OFFSET) @Stable public int nmethodEntryOffset; + @HotSpotVMField(name = "nmethod::_comp_level", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int nmethodCompLevelOffset; + + @HotSpotVMConstant(name = "CompLevel_full_optimization") @Stable public int compilationLevelFullOptimization; + @HotSpotVMConstant(name = "InvocationEntryBci") @Stable public int invocationEntryBci; @HotSpotVMField(name = "JVMCIEnv::_task", type = "CompileTask*", get = HotSpotVMField.Type.OFFSET) @Stable public int jvmciEnvTaskOffset; @HotSpotVMField(name = "JVMCIEnv::_jvmti_can_hotswap_or_post_breakpoint", type = "bool", get = HotSpotVMField.Type.OFFSET) @Stable public int jvmciEnvJvmtiCanHotswapOrPostBreakpointOffset; @HotSpotVMField(name = "CompileTask::_num_inlined_bytecodes", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int compileTaskNumInlinedBytecodesOffset; - /** - * See {@code Method::extra_stack_entries()}. - */ - @HotSpotVMConstant(name = "Method::extra_stack_entries_for_jsr292") @Stable public int extraStackEntries; + @HotSpotVMField(name = "CompilerToVM::Data::Method_extra_stack_entries", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int extraStackEntries; @HotSpotVMField(name = "ConstMethod::_constants", type = "ConstantPool*", get = HotSpotVMField.Type.OFFSET) @Stable public int constMethodConstantsOffset; @HotSpotVMField(name = "ConstMethod::_flags", type = "u2", get = HotSpotVMField.Type.OFFSET) @Stable public int constMethodFlagsOffset; @@ -1325,72 +1327,39 @@ public class HotSpotVMConfig { @HotSpotVMConstant(name = "HeapWordSize") @Stable public int heapWordSize; @HotSpotVMType(name = "Symbol*", get = HotSpotVMType.Type.SIZE) @Stable public int symbolPointerSize; - @HotSpotVMField(name = "Symbol::_length", type = "unsigned short", get = HotSpotVMField.Type.OFFSET) @Stable public int symbolLengthOffset; - @HotSpotVMField(name = "Symbol::_body[0]", type = "jbyte", get = HotSpotVMField.Type.OFFSET) @Stable public int symbolBodyOffset; @HotSpotVMField(name = "vmSymbols::_symbols[0]", type = "Symbol*", get = HotSpotVMField.Type.ADDRESS) @Stable public long vmSymbolsSymbols; @HotSpotVMConstant(name = "vmSymbols::FIRST_SID") @Stable public int vmSymbolsFirstSID; @HotSpotVMConstant(name = "vmSymbols::SID_LIMIT") @Stable public int vmSymbolsSIDLimit; - @HotSpotVMConstant(name = "JVM_ACC_HAS_FINALIZER") @Stable public int klassHasFinalizerFlag; - - // Modifier.SYNTHETIC is not public so we get it via vmStructs. - @HotSpotVMConstant(name = "JVM_ACC_SYNTHETIC") @Stable public int syntheticFlag; - - /** - * @see HotSpotResolvedObjectTypeImpl#createField - */ - @HotSpotVMConstant(name = "JVM_RECOGNIZED_FIELD_MODIFIERS") @Stable public int recognizedFieldModifiers; - /** * Bit pattern that represents a non-oop. Neither the high bits nor the low bits of this value * are allowed to look like (respectively) the high or low bits of a real oop. */ - @HotSpotVMField(name = "Universe::_non_oop_bits", type = "intptr_t", get = HotSpotVMField.Type.VALUE) @Stable public long nonOopBits; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_non_oop_bits", type = "void*", get = HotSpotVMField.Type.VALUE) @Stable public long nonOopBits; @HotSpotVMField(name = "StubRoutines::_verify_oop_count", type = "jint", get = HotSpotVMField.Type.ADDRESS) @Stable public long verifyOopCounterAddress; - @HotSpotVMField(name = "Universe::_verify_oop_mask", type = "uintptr_t", get = HotSpotVMField.Type.VALUE) @Stable public long verifyOopMask; - @HotSpotVMField(name = "Universe::_verify_oop_bits", type = "uintptr_t", get = HotSpotVMField.Type.VALUE) @Stable public long verifyOopBits; - @HotSpotVMField(name = "Universe::_base_vtable_size", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int universeBaseVtableSize; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_verify_oop_mask", type = "uintptr_t", get = HotSpotVMField.Type.VALUE) @Stable public long verifyOopMask; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_verify_oop_bits", type = "uintptr_t", get = HotSpotVMField.Type.VALUE) @Stable public long verifyOopBits; + @HotSpotVMField(name = "CompilerToVM::Data::Universe_base_vtable_size", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int universeBaseVtableSize; public final int baseVtableLength() { return universeBaseVtableSize / vtableEntrySize; } - @HotSpotVMField(name = "CollectedHeap::_barrier_set", type = "BarrierSet*", get = HotSpotVMField.Type.OFFSET) @Stable public int collectedHeapBarrierSetOffset; - @HotSpotVMField(name = "HeapRegion::LogOfHRGrainBytes", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int logOfHRGrainBytes; - @HotSpotVMField(name = "BarrierSet::_fake_rtti", type = "BarrierSet::FakeRtti", get = HotSpotVMField.Type.OFFSET) @Stable private int barrierSetFakeRttiOffset; - @HotSpotVMConstant(name = "BarrierSet::CardTableModRef") @Stable public int barrierSetCardTableModRef; - @HotSpotVMConstant(name = "BarrierSet::CardTableForRS") @Stable public int barrierSetCardTableForRS; - @HotSpotVMConstant(name = "BarrierSet::CardTableExtension") @Stable public int barrierSetCardTableExtension; - @HotSpotVMConstant(name = "BarrierSet::G1SATBCT") @Stable public int barrierSetG1SATBCT; - @HotSpotVMConstant(name = "BarrierSet::G1SATBCTLogging") @Stable public int barrierSetG1SATBCTLogging; - @HotSpotVMConstant(name = "BarrierSet::ModRef") @Stable public int barrierSetModRef; - - @HotSpotVMField(name = "BarrierSet::FakeRtti::_concrete_tag", type = "BarrierSet::Name", get = HotSpotVMField.Type.OFFSET) @Stable private int fakeRttiConcreteTagOffset; - - @HotSpotVMField(name = "CardTableModRefBS::byte_map_base", type = "jbyte*", get = HotSpotVMField.Type.OFFSET) @Stable private int cardTableModRefBSByteMapBaseOffset; - @HotSpotVMConstant(name = "CardTableModRefBS::card_shift") @Stable public int cardTableModRefBSCardShift; - @HotSpotVMConstant(name = "CardTableModRefBS::dirty_card") @Stable public byte dirtyCardValue; @HotSpotVMConstant(name = "G1SATBCardTableModRefBS::g1_young_gen") @Stable public byte g1YoungCardValue; - private final long cardtableStartAddress; - private final int cardtableShift; + @HotSpotVMField(name = "CompilerToVM::Data::cardtable_start_address", type = "jbyte*", get = HotSpotVMField.Type.VALUE) @Stable private long cardtableStartAddress; + @HotSpotVMField(name = "CompilerToVM::Data::cardtable_shift", type = "int", get = HotSpotVMField.Type.VALUE) @Stable private int cardtableShift; public long cardtableStartAddress() { - if (cardtableStartAddress == -1) { - throw JVMCIError.shouldNotReachHere(); - } return cardtableStartAddress; } public int cardtableShift() { - if (cardtableShift == -1) { - throw JVMCIError.shouldNotReachHere(); - } return cardtableShift; } @@ -1421,34 +1390,12 @@ public class HotSpotVMConfig { @HotSpotVMField(name = "java_lang_Class::_klass_offset", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int klassOffset; @HotSpotVMField(name = "java_lang_Class::_array_klass_offset", type = "int", get = HotSpotVMField.Type.VALUE) @Stable public int arrayKlassOffset; - @HotSpotVMField(name = "Method::_method_counters", type = "MethodCounters*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCountersOffset; - @HotSpotVMField(name = "Method::_method_data", type = "MethodData*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOffset; - @HotSpotVMField(name = "Method::_from_compiled_entry", type = "address", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCompiledEntryOffset; - @HotSpotVMField(name = "Method::_code", type = "nmethod*", get = HotSpotVMField.Type.OFFSET) @Stable public int methodCodeOffset; - - @HotSpotVMField(name = "MethodCounters::_invocation_counter", type = "InvocationCounter", get = HotSpotVMField.Type.OFFSET) @Stable public int invocationCounterOffset; - @HotSpotVMField(name = "MethodCounters::_backedge_counter", type = "InvocationCounter", get = HotSpotVMField.Type.OFFSET) @Stable public int backedgeCounterOffset; - @HotSpotVMConstant(name = "InvocationCounter::count_increment") @Stable public int invocationCounterIncrement; - @HotSpotVMConstant(name = "InvocationCounter::count_shift") @Stable public int invocationCounterShift; - - @HotSpotVMField(name = "MethodData::_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataSize; - @HotSpotVMField(name = "MethodData::_data_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataDataSize; - @HotSpotVMField(name = "MethodData::_data[0]", type = "intptr_t", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOopDataOffset; - @HotSpotVMField(name = "MethodData::_trap_hist._array[0]", type = "u1", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataOopTrapHistoryOffset; - @HotSpotVMField(name = "MethodData::_jvmci_ir_size", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int methodDataIRSizeOffset; - - @HotSpotVMField(name = "nmethod::_verified_entry_point", type = "address", get = HotSpotVMField.Type.OFFSET) @Stable public int nmethodEntryOffset; - @HotSpotVMField(name = "nmethod::_comp_level", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable public int nmethodCompLevelOffset; - - @HotSpotVMConstant(name = "CompLevel_full_optimization") @Stable public int compilationLevelFullOptimization; - @HotSpotVMType(name = "BasicLock", get = HotSpotVMType.Type.SIZE) @Stable public int basicLockSize; @HotSpotVMField(name = "BasicLock::_displaced_header", type = "markOop", get = HotSpotVMField.Type.OFFSET) @Stable public int basicLockDisplacedHeaderOffset; @HotSpotVMField(name = "Thread::_allocated_bytes", type = "jlong", get = HotSpotVMField.Type.OFFSET) @Stable public int threadAllocatedBytesOffset; @HotSpotVMFlag(name = "TLABWasteIncrement") @Stable public int tlabRefillWasteIncrement; - @HotSpotVMManual(name = "ThreadLocalAllocBuffer::alignment_reserve()") @Stable public int tlabAlignmentReserve; @HotSpotVMField(name = "ThreadLocalAllocBuffer::_start", type = "HeapWord*", get = HotSpotVMField.Type.OFFSET) @Stable private int threadLocalAllocBufferStartOffset; @HotSpotVMField(name = "ThreadLocalAllocBuffer::_end", type = "HeapWord*", get = HotSpotVMField.Type.OFFSET) @Stable private int threadLocalAllocBufferEndOffset; @@ -1496,22 +1443,14 @@ public class HotSpotVMConfig { return threadTlabOffset + threadLocalAllocBufferPfTopOffset; } - /** - * See: {@code ThreadLocalAllocBuffer::end_reserve()}. - */ - public final int threadLocalAllocBufferEndReserve() { - final int typeSizeInBytes = roundUp(arrayOopDescLengthOffset() + Integer.BYTES, heapWordSize); - // T_INT arrays need not be 8 byte aligned. - final int reserveSize = typeSizeInBytes / heapWordSize; - return Integer.max(reserveSize, abstractVmVersionReserveForAllocationPrefetch); - } + @HotSpotVMField(name = "CompilerToVM::Data::ThreadLocalAllocBuffer_alignment_reserve", type = "size_t", get = HotSpotVMField.Type.VALUE) @Stable public int tlabAlignmentReserve; @HotSpotVMFlag(name = "TLABStats") @Stable public boolean tlabStats; // FIXME This is only temporary until the GC code is changed. - @HotSpotVMField(name = "CompilerToVM::_supports_inline_contig_alloc", type = "bool", get = HotSpotVMField.Type.VALUE) @Stable public boolean inlineContiguousAllocationSupported; - @HotSpotVMField(name = "CompilerToVM::_heap_end_addr", type = "HeapWord**", get = HotSpotVMField.Type.VALUE) @Stable public long heapEndAddress; - @HotSpotVMField(name = "CompilerToVM::_heap_top_addr", type = "HeapWord**", get = HotSpotVMField.Type.VALUE) @Stable public long heapTopAddress; + @HotSpotVMField(name = "CompilerToVM::Data::_supports_inline_contig_alloc", type = "bool", get = HotSpotVMField.Type.VALUE) @Stable public boolean inlineContiguousAllocationSupported; + @HotSpotVMField(name = "CompilerToVM::Data::_heap_end_addr", type = "HeapWord**", get = HotSpotVMField.Type.VALUE) @Stable public long heapEndAddress; + @HotSpotVMField(name = "CompilerToVM::Data::_heap_top_addr", type = "HeapWord**", get = HotSpotVMField.Type.VALUE) @Stable public long heapTopAddress; /** * The DataLayout header size is the same as the cell size. @@ -1542,19 +1481,11 @@ public class HotSpotVMConfig { @HotSpotVMFlag(name = "TypeProfileWidth") @Stable public int typeProfileWidth; @HotSpotVMFlag(name = "MethodProfileWidth") @Stable public int methodProfileWidth; - @HotSpotVMField(name = "CodeBlob::_code_offset", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable private int codeBlobCodeOffsetOffset; - @HotSpotVMField(name = "DeoptimizationBlob::_unpack_offset", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable private int deoptimizationBlobUnpackOffsetOffset; - @HotSpotVMField(name = "DeoptimizationBlob::_uncommon_trap_offset", type = "int", get = HotSpotVMField.Type.OFFSET) @Stable private int deoptimizationBlobUncommonTrapOffsetOffset; + @HotSpotVMField(name = "CompilerToVM::Data::SharedRuntime_ic_miss_stub", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long inlineCacheMissStub; + @HotSpotVMField(name = "CompilerToVM::Data::SharedRuntime_handle_wrong_method_stub", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long handleWrongMethodStub; - @HotSpotVMField(name = "SharedRuntime::_ic_miss_blob", type = "RuntimeStub*", get = HotSpotVMField.Type.VALUE) @Stable private long inlineCacheMissBlob; - @HotSpotVMField(name = "SharedRuntime::_wrong_method_blob", type = "RuntimeStub*", get = HotSpotVMField.Type.VALUE) @Stable private long wrongMethodBlob; - @HotSpotVMField(name = "SharedRuntime::_deopt_blob", type = "DeoptimizationBlob*", get = HotSpotVMField.Type.VALUE) @Stable private long deoptBlob; - - @HotSpotVMManual(name = "SharedRuntime::get_ic_miss_stub()") public final long inlineCacheMissStub; - @HotSpotVMManual(name = "SharedRuntime::get_handle_wrong_method_stub()") public final long handleWrongMethodStub; - - @HotSpotVMManual(name = "SharedRuntime::deopt_blob()->unpack()") public final long handleDeoptStub; - @HotSpotVMManual(name = "SharedRuntime::deopt_blob()->uncommon_trap()") public final long uncommonTrapStub; + @HotSpotVMField(name = "CompilerToVM::Data::SharedRuntime_deopt_blob_unpack", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long handleDeoptStub; + @HotSpotVMField(name = "CompilerToVM::Data::SharedRuntime_deopt_blob_uncommon_trap", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long uncommonTrapStub; @HotSpotVMField(name = "CodeCache::_low_bound", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long codeCacheLowBound; @HotSpotVMField(name = "CodeCache::_high_bound", type = "address", get = HotSpotVMField.Type.VALUE) @Stable public long codeCacheHighBound; @@ -1717,9 +1648,6 @@ public class HotSpotVMConfig { return "unknown"; } - @HotSpotVMConstant(name = "CompilerToVM::KLASS_TAG") @Stable public int compilerToVMKlassTag; - @HotSpotVMConstant(name = "CompilerToVM::SYMBOL_TAG") @Stable public int compilerToVMSymbolTag; - // Checkstyle: stop @HotSpotVMConstant(name = "CodeInstaller::VERIFIED_ENTRY") @Stable public int MARKID_VERIFIED_ENTRY; @HotSpotVMConstant(name = "CodeInstaller::UNVERIFIED_ENTRY") @Stable public int MARKID_UNVERIFIED_ENTRY; diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/Stable.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/Stable.java index e386dc0ca77..f313238bf41 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/Stable.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/Stable.java @@ -29,7 +29,7 @@ import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; /** - * This annotation functions as an alias for the java.lang.invoke.Stable annotation within JVMCI + * This annotation functions as an alias for the jdk.internal.vm.annotation.Stable annotation within JVMCI * code. It is specially recognized during class file parsing in the same way as that annotation. */ @Target(ElementType.FIELD) diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.inittimer/src/jdk/vm/ci/inittimer/InitTimer.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.inittimer/src/jdk/vm/ci/inittimer/InitTimer.java index 608285e7e92..921b537bfb1 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.inittimer/src/jdk/vm/ci/inittimer/InitTimer.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.inittimer/src/jdk/vm/ci/inittimer/InitTimer.java @@ -22,6 +22,8 @@ */ package jdk.vm.ci.inittimer; +import java.util.concurrent.atomic.AtomicInteger; + /** * A facility for timing a step in the runtime initialization sequence. This is independent from all * other JVMCI code so as to not perturb the initialization sequence. It is enabled by setting the @@ -32,18 +34,26 @@ public final class InitTimer implements AutoCloseable { final long start; private InitTimer(String name) { + int n = nesting.getAndIncrement(); + if (n == 0) { + initializingThread = Thread.currentThread(); + System.out.println("INITIALIZING THREAD: " + initializingThread); + } else { + assert Thread.currentThread() == initializingThread : Thread.currentThread() + " != " + initializingThread; + } this.name = name; this.start = System.currentTimeMillis(); - System.out.println("START: " + SPACES.substring(0, timerDepth * 2) + name); - assert Thread.currentThread() == initializingThread : Thread.currentThread() + " != " + initializingThread; - timerDepth++; + System.out.println("START: " + SPACES.substring(0, n * 2) + name); } @SuppressFBWarnings(value = "ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD", justification = "only the initializing thread accesses this field") public void close() { final long end = System.currentTimeMillis(); - timerDepth--; - System.out.println(" DONE: " + SPACES.substring(0, timerDepth * 2) + name + " [" + (end - start) + " ms]"); + int n = nesting.decrementAndGet(); + System.out.println(" DONE: " + SPACES.substring(0, n * 2) + name + " [" + (end - start) + " ms]"); + if (n == 0) { + initializingThread = null; + } } public static InitTimer timer(String name) { @@ -59,19 +69,11 @@ public final class InitTimer implements AutoCloseable { */ private static final boolean ENABLED = Boolean.getBoolean("jvmci.inittimer") || Boolean.getBoolean("jvmci.runtime.TimeInit"); - public static int timerDepth = 0; + public static final AtomicInteger nesting = ENABLED ? new AtomicInteger() : null; public static final String SPACES = " "; /** - * Used to assert the invariant that all initialization happens on the same thread. + * Used to assert the invariant that all related initialization happens on the same thread. */ - public static final Thread initializingThread; - static { - if (ENABLED) { - initializingThread = Thread.currentThread(); - System.out.println("INITIALIZING THREAD: " + initializingThread); - } else { - initializingThread = null; - } - } + public static Thread initializingThread; } diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.meta/src/jdk/vm/ci/meta/ConstantReflectionProvider.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.meta/src/jdk/vm/ci/meta/ConstantReflectionProvider.java index 1db2b98c7ea..33cc2a27ebc 100644 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.meta/src/jdk/vm/ci/meta/ConstantReflectionProvider.java +++ b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.meta/src/jdk/vm/ci/meta/ConstantReflectionProvider.java @@ -80,29 +80,31 @@ public interface ConstantReflectionProvider { * @return the constant value of this field or {@code null} if this field is not considered * constant by the runtime */ - JavaConstant readConstantFieldValue(JavaField field, JavaConstant receiver); + JavaConstant readConstantFieldValue(ResolvedJavaField field, JavaConstant receiver); /** * Gets the current value of this field for a given object, if available. * * There is no guarantee that the same value will be returned by this method for a field unless - * the field is considered to be {@linkplain #readConstantFieldValue(JavaField, JavaConstant) - * constant} by the runtime. + * the field is considered to be + * {@linkplain #readConstantFieldValue(ResolvedJavaField, JavaConstant) constant} by the + * runtime. * * @param receiver object from which this field's value is to be read. This value is ignored if * this field is static. * @return the value of this field or {@code null} if the value is not available (e.g., because * the field holder is not yet initialized). */ - JavaConstant readFieldValue(JavaField field, JavaConstant receiver); + JavaConstant readFieldValue(ResolvedJavaField field, JavaConstant receiver); /** * Gets the current value of this field for a given object, if available. Like - * {@link #readFieldValue(JavaField, JavaConstant)} but treats array fields as stable. + * {@link #readFieldValue(ResolvedJavaField, JavaConstant)} but treats array fields as stable. * * There is no guarantee that the same value will be returned by this method for a field unless - * the field is considered to be {@linkplain #readConstantFieldValue(JavaField, JavaConstant) - * constant} by the runtime. + * the field is considered to be + * {@linkplain #readConstantFieldValue(ResolvedJavaField, JavaConstant) constant} by the + * runtime. * * @param receiver object from which this field's value is to be read. This value is ignored if * this field is static. @@ -110,7 +112,7 @@ public interface ConstantReflectionProvider { * @return the value of this field or {@code null} if the value is not available (e.g., because * the field holder is not yet initialized). */ - JavaConstant readStableFieldValue(JavaField field, JavaConstant receiver, boolean isDefaultStable); + JavaConstant readStableFieldValue(ResolvedJavaField field, JavaConstant receiver, boolean isDefaultStable); /** * Converts the given {@link JavaKind#isPrimitive() primitive} constant to a boxed diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/META-INF/services/javax.annotation.processing.Processor b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/META-INF/services/javax.annotation.processing.Processor deleted file mode 100644 index 3aab30fa3c7..00000000000 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/META-INF/services/javax.annotation.processing.Processor +++ /dev/null @@ -1 +0,0 @@ -jdk.vm.ci.options.processor.OptionProcessor diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/jdk/vm/ci/options/processor/OptionProcessor.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/jdk/vm/ci/options/processor/OptionProcessor.java deleted file mode 100644 index 95422c24135..00000000000 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options.processor/src/jdk/vm/ci/options/processor/OptionProcessor.java +++ /dev/null @@ -1,369 +0,0 @@ -/* - * Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved. - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This code is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 only, as - * published by the Free Software Foundation. - * - * This code is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License - * version 2 for more details (a copy is included in the LICENSE file that - * accompanied this code). - * - * You should have received a copy of the GNU General Public License version - * 2 along with this work; if not, write to the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. - * - * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA - * or visit www.oracle.com if you need additional information or have any - * questions. - */ -package jdk.vm.ci.options.processor; - -import java.io.IOException; -import java.io.PrintWriter; -import java.util.ArrayList; -import java.util.Collections; -import java.util.HashMap; -import java.util.HashSet; -import java.util.List; -import java.util.Map; -import java.util.Set; - -import javax.annotation.processing.AbstractProcessor; -import javax.annotation.processing.Filer; -import javax.annotation.processing.RoundEnvironment; -import javax.annotation.processing.SupportedAnnotationTypes; -import javax.lang.model.SourceVersion; -import javax.lang.model.element.Element; -import javax.lang.model.element.ElementKind; -import javax.lang.model.element.Modifier; -import javax.lang.model.element.Name; -import javax.lang.model.element.PackageElement; -import javax.lang.model.element.TypeElement; -import javax.lang.model.element.VariableElement; -import javax.lang.model.type.DeclaredType; -import javax.lang.model.type.TypeKind; -import javax.lang.model.type.TypeMirror; -import javax.lang.model.util.Elements; -import javax.lang.model.util.Types; -import javax.tools.Diagnostic.Kind; -import javax.tools.JavaFileObject; - -import jdk.vm.ci.options.Option; -import jdk.vm.ci.options.OptionDescriptor; -import jdk.vm.ci.options.OptionDescriptors; -import jdk.vm.ci.options.OptionValue; - -/** - * Processes static fields annotated with {@link Option}. An {@link OptionDescriptors} - * implementation is generated for each top level class containing at least one such field. The name - * of the generated class for top level class {@code com.foo.Bar} is - * {@code com.foo.Bar_OptionDescriptors}. - */ -@SupportedAnnotationTypes({"jdk.vm.ci.options.Option"}) -public class OptionProcessor extends AbstractProcessor { - - @Override - public SourceVersion getSupportedSourceVersion() { - return SourceVersion.latest(); - } - - private final Set processed = new HashSet<>(); - - private void processElement(Element element, OptionsInfo info) { - - if (!element.getModifiers().contains(Modifier.STATIC)) { - processingEnv.getMessager().printMessage(Kind.ERROR, "Option field must be static", element); - return; - } - - Option annotation = element.getAnnotation(Option.class); - assert annotation != null; - assert element instanceof VariableElement; - assert element.getKind() == ElementKind.FIELD; - VariableElement field = (VariableElement) element; - String fieldName = field.getSimpleName().toString(); - - Elements elements = processingEnv.getElementUtils(); - Types types = processingEnv.getTypeUtils(); - - TypeMirror fieldType = field.asType(); - if (fieldType.getKind() != TypeKind.DECLARED) { - processingEnv.getMessager().printMessage(Kind.ERROR, "Option field must be of type " + OptionValue.class.getName(), element); - return; - } - DeclaredType declaredFieldType = (DeclaredType) fieldType; - - TypeMirror optionValueType = elements.getTypeElement(OptionValue.class.getName()).asType(); - if (!types.isSubtype(fieldType, types.erasure(optionValueType))) { - String msg = String.format("Option field type %s is not a subclass of %s", fieldType, optionValueType); - processingEnv.getMessager().printMessage(Kind.ERROR, msg, element); - return; - } - - if (!field.getModifiers().contains(Modifier.STATIC)) { - processingEnv.getMessager().printMessage(Kind.ERROR, "Option field must be static", element); - return; - } - - String help = annotation.help(); - if (help.length() != 0) { - char firstChar = help.charAt(0); - if (!Character.isUpperCase(firstChar)) { - processingEnv.getMessager().printMessage(Kind.ERROR, "Option help text must start with upper case letter", element); - return; - } - } - - String optionName = annotation.name(); - if (optionName.equals("")) { - optionName = fieldName; - } - - DeclaredType declaredOptionValueType = declaredFieldType; - while (!types.isSameType(types.erasure(declaredOptionValueType), types.erasure(optionValueType))) { - List directSupertypes = types.directSupertypes(declaredFieldType); - assert !directSupertypes.isEmpty(); - declaredOptionValueType = (DeclaredType) directSupertypes.get(0); - } - - assert !declaredOptionValueType.getTypeArguments().isEmpty(); - String optionType = declaredOptionValueType.getTypeArguments().get(0).toString(); - if (optionType.startsWith("java.lang.")) { - optionType = optionType.substring("java.lang.".length()); - } - - Element enclosing = element.getEnclosingElement(); - String declaringClass = ""; - String separator = ""; - Set originatingElementsList = info.originatingElements; - originatingElementsList.add(field); - while (enclosing != null) { - if (enclosing.getKind() == ElementKind.CLASS || enclosing.getKind() == ElementKind.INTERFACE) { - if (enclosing.getModifiers().contains(Modifier.PRIVATE)) { - String msg = String.format("Option field cannot be declared in a private %s %s", enclosing.getKind().name().toLowerCase(), enclosing); - processingEnv.getMessager().printMessage(Kind.ERROR, msg, element); - return; - } - originatingElementsList.add(enclosing); - declaringClass = enclosing.getSimpleName() + separator + declaringClass; - separator = "."; - } else { - assert enclosing.getKind() == ElementKind.PACKAGE; - } - enclosing = enclosing.getEnclosingElement(); - } - - info.options.add(new OptionInfo(optionName, help, optionType, declaringClass, field)); - } - - private void createFiles(OptionsInfo info) { - String pkg = ((PackageElement) info.topDeclaringType.getEnclosingElement()).getQualifiedName().toString(); - Name topDeclaringClass = info.topDeclaringType.getSimpleName(); - Element[] originatingElements = info.originatingElements.toArray(new Element[info.originatingElements.size()]); - - createOptionsDescriptorsFile(info, pkg, topDeclaringClass, originatingElements); - } - - private void createOptionsDescriptorsFile(OptionsInfo info, String pkg, Name topDeclaringClass, Element[] originatingElements) { - String optionsClassName = topDeclaringClass + "_" + OptionDescriptors.class.getSimpleName(); - - Filer filer = processingEnv.getFiler(); - try (PrintWriter out = createSourceFile(pkg, optionsClassName, filer, originatingElements)) { - - out.println("// CheckStyle: stop header check"); - out.println("// CheckStyle: stop line length check"); - out.println("// GENERATED CONTENT - DO NOT EDIT"); - out.println("// Source: " + topDeclaringClass + ".java"); - out.println("package " + pkg + ";"); - out.println(""); - out.println("import java.util.*;"); - out.println("import " + OptionDescriptors.class.getPackage().getName() + ".*;"); - out.println(""); - out.println("public class " + optionsClassName + " implements " + OptionDescriptors.class.getSimpleName() + " {"); - - String desc = OptionDescriptor.class.getSimpleName(); - - boolean needPrivateFieldAccessor = false; - int i = 0; - Collections.sort(info.options); - - out.println(" @Override"); - out.println(" public OptionDescriptor get(String value) {"); - out.println(" // CheckStyle: stop line length check"); - if (info.options.size() == 1) { - out.println(" if (value.equals(\"" + info.options.get(0).name + "\")) {"); - } else { - out.println(" switch (value) {"); - } - for (OptionInfo option : info.options) { - String name = option.name; - String optionValue; - if (option.field.getModifiers().contains(Modifier.PRIVATE)) { - needPrivateFieldAccessor = true; - optionValue = "field(" + option.declaringClass + ".class, \"" + option.field.getSimpleName() + "\")"; - } else { - optionValue = option.declaringClass + "." + option.field.getSimpleName(); - } - String type = option.type; - String help = option.help; - String declaringClass = option.declaringClass; - Name fieldName = option.field.getSimpleName(); - if (info.options.size() == 1) { - out.printf(" return %s.create(\"%s\", %s.class, \"%s\", %s.class, \"%s\", %s);\n", desc, name, type, help, declaringClass, fieldName, optionValue); - } else { - out.printf(" case \"" + name + "\": return %s.create(\"%s\", %s.class, \"%s\", %s.class, \"%s\", %s);\n", desc, name, type, help, declaringClass, fieldName, optionValue); - } - } - out.println(" }"); - out.println(" // CheckStyle: resume line length check"); - out.println(" return null;"); - out.println(" }"); - out.println(); - out.println(" @Override"); - out.println(" public Iterator<" + desc + "> iterator() {"); - out.println(" // CheckStyle: stop line length check"); - out.println(" List<" + desc + "> options = Arrays.asList("); - for (OptionInfo option : info.options) { - String optionValue; - if (option.field.getModifiers().contains(Modifier.PRIVATE)) { - needPrivateFieldAccessor = true; - optionValue = "field(" + option.declaringClass + ".class, \"" + option.field.getSimpleName() + "\")"; - } else { - optionValue = option.declaringClass + "." + option.field.getSimpleName(); - } - String name = option.name; - String type = option.type; - String help = option.help; - String declaringClass = option.declaringClass; - Name fieldName = option.field.getSimpleName(); - String comma = i == info.options.size() - 1 ? "" : ","; - out.printf(" %s.create(\"%s\", %s.class, \"%s\", %s.class, \"%s\", %s)%s\n", desc, name, type, help, declaringClass, fieldName, optionValue, comma); - i++; - } - out.println(" );"); - out.println(" // CheckStyle: resume line length check"); - out.println(" return options.iterator();"); - out.println(" }"); - if (needPrivateFieldAccessor) { - out.println(" private static " + OptionValue.class.getSimpleName() + " field(Class declaringClass, String fieldName) {"); - out.println(" try {"); - out.println(" java.lang.reflect.Field field = declaringClass.getDeclaredField(fieldName);"); - out.println(" field.setAccessible(true);"); - out.println(" return (" + OptionValue.class.getSimpleName() + ") field.get(null);"); - out.println(" } catch (Exception e) {"); - out.println(" throw (InternalError) new InternalError().initCause(e);"); - out.println(" }"); - out.println(" }"); - } - out.println("}"); - } - } - - protected PrintWriter createSourceFile(String pkg, String relativeName, Filer filer, Element... originatingElements) { - try { - // Ensure Unix line endings to comply with code style guide checked by Checkstyle - JavaFileObject sourceFile = filer.createSourceFile(pkg + "." + relativeName, originatingElements); - return new PrintWriter(sourceFile.openWriter()) { - - @Override - public void println() { - print("\n"); - } - }; - } catch (IOException e) { - throw new RuntimeException(e); - } - } - - static class OptionInfo implements Comparable { - - final String name; - final String help; - final String type; - final String declaringClass; - final VariableElement field; - - public OptionInfo(String name, String help, String type, String declaringClass, VariableElement field) { - this.name = name; - this.help = help; - this.type = type; - this.declaringClass = declaringClass; - this.field = field; - } - - @Override - public int compareTo(OptionInfo other) { - return name.compareTo(other.name); - } - - @Override - public String toString() { - return declaringClass + "." + field; - } - } - - static class OptionsInfo { - - final Element topDeclaringType; - final List options = new ArrayList<>(); - final Set originatingElements = new HashSet<>(); - - public OptionsInfo(Element topDeclaringType) { - this.topDeclaringType = topDeclaringType; - } - } - - private static Element topDeclaringType(Element element) { - Element enclosing = element.getEnclosingElement(); - if (enclosing == null || enclosing.getKind() == ElementKind.PACKAGE) { - assert element.getKind() == ElementKind.CLASS || element.getKind() == ElementKind.INTERFACE; - return element; - } - return topDeclaringType(enclosing); - } - - @Override - public boolean process(Set annotations, RoundEnvironment roundEnv) { - if (roundEnv.processingOver()) { - return true; - } - - Map map = new HashMap<>(); - for (Element element : roundEnv.getElementsAnnotatedWith(Option.class)) { - if (!processed.contains(element)) { - processed.add(element); - Element topDeclaringType = topDeclaringType(element); - OptionsInfo options = map.get(topDeclaringType); - if (options == null) { - options = new OptionsInfo(topDeclaringType); - map.put(topDeclaringType, options); - } - processElement(element, options); - } - } - - boolean ok = true; - Map uniqueness = new HashMap<>(); - for (OptionsInfo info : map.values()) { - for (OptionInfo option : info.options) { - OptionInfo conflict = uniqueness.put(option.name, option); - if (conflict != null) { - processingEnv.getMessager().printMessage(Kind.ERROR, "Duplicate option names for " + option + " and " + conflict, option.field); - ok = false; - } - } - } - - if (ok) { - for (OptionsInfo info : map.values()) { - createFiles(info); - } - } - - return true; - } -} diff --git a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options/src/jdk/vm/ci/options/NestedBooleanOptionValue.java b/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options/src/jdk/vm/ci/options/NestedBooleanOptionValue.java deleted file mode 100644 index 804603814f1..00000000000 --- a/hotspot/src/jdk.vm.ci/share/classes/jdk.vm.ci.options/src/jdk/vm/ci/options/NestedBooleanOptionValue.java +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This code is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 only, as - * published by the Free Software Foundation. - * - * This code is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License - * version 2 for more details (a copy is included in the LICENSE file that - * accompanied this code). - * - * You should have received a copy of the GNU General Public License version - * 2 along with this work; if not, write to the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. - * - * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA - * or visit www.oracle.com if you need additional information or have any - * questions. - */ -package jdk.vm.ci.options; - -/** - * A nested Boolean {@link OptionValue} that can be overridden by a {@link #masterOption master - * option}. - *

    - *

  • If the option is present on the command line the specified value is used. - *
  • Otherwise {@link #getValue()} depends on the {@link #masterOption} and evaluates as follows: - *