Java关键字volatile的使用

volatile是在多线程编程中经常需要用到的一个关键字,很多地方都有关于这个关键字的用法说明,但一直看得一知半解,这里汇集一下网上收集到的各种资料,辅以示例代码来说明,便于理解。

Java内存模型与线程

多任务和高并发是衡量一台计算机处理器的能力重要指标之一。一般衡量一个服务器性能的高低好坏,使用每秒事务处理数(Transactions Per Second,TPS)这个指标比较能说明问题,它代表着一秒内服务器平均能响应的请求数,而TPS值与程序的并发能力有着非常密切的关系。

硬件的效率与一致性

由于计算机的存储设备与处理器的运算能力之间有几个数量级的差距,所以现代计算机系统都不得不加入一层读写速度尽可能接近处理器运算速度的高速缓存(cache)来作为内存与处理器之间的缓冲:将运算需要使用到的数据复制到缓存中,让运算能快速进行,当运算结束后再从缓存同步回内存之中没这样处理器就无需等待缓慢的内存读写了。 基于高速缓存的存储交互很好地解决了处理器与内存的速度矛盾,但是引入了一个新的问题:缓存一致性(Cache Coherence)。在多处理器系统中,每个处理器都有自己的高速缓存,而他们又共享同一主存,如下图所示:多个处理器运算任务都涉及同一块主存,需要一种协议可以保障数据的一致性,这类协议有MSI、MESI、MOSI及Dragon Protocol等。Java虚拟机内存模型中定义的内存访问操作与硬件的缓存访问操作是具有可比性的,后续将介绍Java内存模型。

除此之外,为了使得处理器内部的运算单元能竟可能被充分利用,处理器可能会对输入代码进行乱序执行(Out-Of-Order Execution)优化,处理器会在计算之后将对乱序执行的代码进行结果重组,保证结果准确性。与处理器的乱序执行优化类似,Java虚拟机的即时编译器中也有类似的指令重排序(Instruction Recorder)优化。

if (doSomething("上车")) {       
   doSomethingElse("刷卡") 
 } 

计算机实际执行时,可能先刷卡,再判断是否上车,如果上车了,就使用刷卡的结果,如果没上车,就抛弃掉刷卡的结果。

Java 内存模型

JMM(Java Memory Model)试图屏蔽掉各种硬件和操作系统的内存访问差异,以实现让Java程序在各种平台下都能达到一致的并发效果。在JDK1.5后,Java内存模型已经成熟和完善。

主内存与工作内存

Java内存模型的主要目标是定义程序中各个变量的访问规则,即在虚拟机中将变量存储到内存和从内存中取出变量这样底层细节。此处的变量与Java编程时所说的变量不一样,指包括了实例字段、静态字段和构成数组对象的元素,但是不包括局部变量与方法参数,后者是线程私有的,不会被共享。

Java内存模型中规定了所有的变量都存储在主内存中,每条线程还有自己的工作内存(可以与前面将的处理器的高速缓存类比),线程的工作内存中保存了该线程使用到的变量到主内存副本拷贝,线程对变量的所有操作(读取、赋值)都必须在工作内存中进行,而不能直接读写主内存中的变量。不同线程之间无法直接访问对方工作内存中的变量,线程间变量值的传递均需要在主内存来完成,线程、主内存和工作内存的交互关系如下图所示,和上图很类似。

内存间交互操作

一个变量如何从主内存拷贝到工作内存、如何从工作内存同步回主内存之类的实现细节,JMM定义了一下八种操作来完成:

  • lock(锁定):作用域主内存的变量,它把一个变量标识为一条线程独占的状态;
  • unlock(解锁):作用于主内存的变量,它把一个处于锁定状态的变量释放出来,释放后的变量才可以被其他线程锁定;
  • read(读取):作用于主内存变量,它变量的值从主内存传输到线程的工作内存中,以便随后的load动作使用
  • load(载入):作用于工作内存的变量,它把read操作从主内存中得到的变量值放入工作内存的变量副本中;
  • use(使用):作用于工作内存的变量,它把工作内存中一个变量的值传递给执行引擎,每当虚拟机遇到一个需要使用变量的值的字节码指令时将会执行这个操作。
  • assign(赋值):作用于工作内存的变量,它把一个从执行引擎接收到的值赋值给工作内存的变量,每当虚拟机遇到一个给变量赋值的字节码指令时执行这个操作。
  • store(存储):作用于工作内存的变量,把工作内存中的一个变量的值传送到主内存中,以便随后的write的操作。
  • write(写入):作用于主内存的变量,它把store操作从工作内存中一个变量的值传送到主内存的变量中。

如果要把一个变量从主内存中复制到工作内存,就需要按顺寻地执行read和load操作,如果把变量从工作内存中同步回主内存中,就要按顺序地执行store和write操作。Java内存模型只要求上述操作必须按顺序执行,而没有保证必须是连续执行。也就是read和load之间,store和write之间是可以插入其他指令的,如对主内存中的变量a、b进行访问时,可能的顺序是read a,read b,load b, load a。Java内存模型还规定了在执行上述八种基本操作时,必须满足如下规则:

  • 不允许read和load、store和write操作之一单独出现,如不允许从主内存读取了但工作内存不接受
  • 不允许一个线程丢弃它的最近assign的操作,即变量在工作内存中改变了之后必须同步到主内存中。
  • 不允许一个线程无原因地(没有发生过任何assign操作)把数据从工作内存同步回主内存中。
  • 一个新的变量只能在主内存中诞生,不允许在工作内存中直接使用一个未被初始化(load或assign)的变量。即就是对一个变量实施use和store操作之前,必须先执行过了assign和load操作。
  • 一个变量在同一时刻只允许一条线程对其进行lock操作,lock和unlock必须成对出现
  • 如果对一个变量执行lock操作,将会清空工作内存中此变量的值,在执行引擎使用这个变量前需要重新执行load或assign操作初始化变量的值
  • 如果一个变量事先没有被lock操作锁定,则不允许对它执行unlock操作;也不允许去unlock一个被其他线程锁定的变量。
  • 对一个变量执行unlock操作之前,必须先把此变量同步到主内存中(执行store和write操作)。

对于volatile型变量的特殊规则

当一个变量被定义成volatile之后,它将具备两种特性,第一是保证此变量对所有线程的可见性,即当一条线程修改了这个变量的值,新值对于其他线程来说是立即得知的。

由于volatile变量只能保证可见性,在不符合以下两条规则的运算场景中,我们仍然要通过加锁来保证原子性。

  • 运算结果并不依赖变量的当前值,或者能够确保只有单一的线程修改变量的值
  • 变量不需要与其他状态变量共同参与不变约束

volatile变量的第二个语义是禁止指令重排序优化。

编译器只保证程序执行结果与源代码相同,却不保证实际指令的顺序与源代码相同。这在单线程看起来没什么问题,然而一旦引入多线程,这种乱序就可能导致严重问题。volatile关键字就可以从语义上解决这个问题。

例如,考虑下面的事件序列:

  1. 线程A发现变量没有被初始化, 然后它获取锁并开始变量的初始化。
  2. 由于某些编程语言的语义,编译器生成的代码允许在线程A执行完变量的初始化之前,更新变量并将其指向部分初始化的对象。
  3. 线程B发现共享变量已经被初始化,并返回变量。由于线程B确信变量已被初始化,它没有获取锁。如果在A完成初始化之前共享变量对B可见(这是由于A没有完成初始化或者因为一些初始化的值还没有穿过B使用的内存(缓存一致性)),程序很可能会崩溃。
public class Singleton {
    private static volatile Singleton singleton = null;
 
    private Singleton(){}
 
    public static Singleton getSingleton(){
        if(singleton == null){
            synchronized (Singleton.class){
                if(singleton == null){
                    singleton = new Singleton();
                }
            }
        }
        return singleton;
    }   
}

Symantec JIT 编译 singletons[i].reference = new Singleton(); 这段代码时,如果不加volatile关键词,会生成如下字节码:

; allocate space for  Singleton, return result in eax 
0206106A   mov         eax,0F97E78h 
0206106F   call        01F6B210                  

; EBP is &singletons[i].reference, store the unconstructed object here. 
02061074   mov         dword ptr [ebp],eax  
            
; dereference the handle to get the raw pointer 
02061077   mov         ecx,dword ptr [eax] 

; Next 4 lines are Singleton's inlined constructor 
02061079   mov         dword ptr [ecx],100h      
0206107F   mov         dword ptr [ecx+4],200h 
02061086   mov         dword ptr [ecx+8],400h
0206108D   mov         dword ptr [ecx+0Ch],0F84030h

可以看到,在执行Singleton的构造函数之前,Singleton的新实例就被赋值给了singletons[i].reference,这在Java内存模型中是完全合法的。

对于long和double型变量的特殊规则

JVM规范允许虚拟机将没有被volatile修饰的64位数据的读写操作划分为两次32位的操作来进行。但是各种虚拟机实现几乎把64位数据的读写作为原子操作来对待(如果分两次来操作,极端情况下得到的结果就会很诡异)

原子性、可见性和有序性

  • 原子性(Atomicity):大致认为基本数据类型的访问读写是具备原子性的。JMM提供lock和unlock保证原子性,对应代码中的synchronized关键字
  • 可见性(Visibility):当一个线程修改了共享变量的值,其他线程能够立即得知这个修改。volatile保证新值能立即同步到主内存,以及每次使用前立即从主内存刷新。除了volatile外,synchronized和final两个关键字也能实现可见性,其中同步块是有lock和unlock机制决定的,而final关键字一旦初始化完成,其他线程就能看见final字段的值
  • 有序性(Ordering):如果在本线程内观察,所有的操作都是有序的;如果在一个线程内观察另一个线程,所有操作都是无序的。Java提供了volatile和synchronized来听歌关键字来保证线程之间操作的有序性。

先行发生原则( happens-before )

先行发生原则:如果操作A先发生于操作B,操作A产生的影响能被操作B观察到,“影响”包括:修改了内存中共享变量的值、发送了消息、调用了方法。

  • 程序次序规则:写在程序签名的操作先行发生于书写在后面的操作
  • 管程锁定规则:一个unlock操作先行发生于后面对同一个锁的lock操作
  • volatile变量规则:对一个volatile变量的写操作先行发生于后面对这个变量的读操作
  • 线程启动规则:Thread对象的start方法先行发生于此线程的每一个动作
  • 线程终止规则:线程中的所有操作都先行发生于对此线程的终止检测
  • 线程中断规则:对线程interrupt()方法的调用先行发生于被中断线程的代码检测到中断事件的发生
  • 对象终结规则:一个对象初始化完成先行发生于它的finalize方法的开始
  • 传递性:如果操作A先于操作B,操作B先行于操作C,那么操作A先行发生于操作C

来看一段代码:

public class PrintString implements Runnable{
    private boolean isRunnning = true;

    @Override
    public void run() {
        System.out.println("Thread begin: "+Thread.currentThread().getName());
        while (isRunnning == true){
        }
        System.out.println("Thread end: "+Thread.currentThread().getName());
    }

    public boolean isRunnning() {
        return isRunnning;
    }

    public void setRunnning(boolean runnning) {
        isRunnning = runnning;
    }

    public static void main(String[] args) throws InterruptedException {
        PrintString printString = new PrintString();
        Thread thread = new Thread(printString,"Thread-A");
        thread.start();
        Thread.sleep(1000);
        printString.setRunnning(false);
        System.out.println(" 我要停止它!" + Thread.currentThread().getName());
    }
}

JVM 有 Client 和 Server 两种模式,我们可以通过运行:java -version 来查看 jvm 默认工作在什么模式。我们在 IDE 中把 JVM 设置为在 Server 服务器的环境中,具体操作只需配置运行参数为 -server。然后启动程序,打印结果:

Thread begin: Thread-A
我要停止它!main

代码 System.out.println(“Thread end: “+Thread.currentThread().getName()); 从未被执行。

是什么样的原因造成将 JVM 设置为 -server 就出现死循环呢?

在启动 thread 线程时,变量 boolean isContinuePrint = true; 存在于公共堆栈及线程的私有堆栈中。在 JVM 设置为 -server 模式时为了线程运行的效率,线程一直在私有堆栈中取得 isRunning 的值是 true。而代码 thread.setRunning(false); 虽然被执行,更新的却是公共堆栈中的 isRunning 变量值 false,所以一直就是死循环的状态。

将代码更改如下:

volatile private boolean isRunnning = true;

再次运行:

Thread begin: Thread-A
我要停止它!main
Thread end: Thread-A

通过使用 volatile 关键字,强制的从公共内存中读取变量的值。

到此基本上了解 volatile的使用了,再来看一个有趣的事情:

public class Test {
    private boolean isRunnning = true;

    public static void main(String[] args) throws Exception {
        PrintString printString = new PrintString();
        Thread thread = new Thread(printString,"Thread-A");
        thread.start();
        Thread.sleep(1000);
        System.out.println(" 我要停止它!" + Thread.currentThread().getName());
        printString.setRunnning(false);
    }

    public boolean isRunnning() {
        return isRunnning;
    }

    public void setRunnning(boolean runnning) {
        isRunnning = runnning;
    }
}

class PrintString implements Runnable{
    private volatile Test test = new Test();

    @Override
    public void run() {
        System.out.println("Thread begin: "+Thread.currentThread().getName());
        while (test.isRunnning()){
        }
        System.out.println("Thread end: "+Thread.currentThread().getName());
    }

    public void setRunnning(boolean runnning) {
        test.setRunnning(runnning);
    }

    public void setTest(Test test) {
        this.test = test;
    }
}

Test 里面的isRunnning 没加volatile 关键字,但PrintString 里面的test 变量加了,程序能正常结束,如果都不加 volatile ,程序会死循环,JVM在缓存变量时,会把变量相关的引用变量一起缓存,但对于 volatile 修饰的变量,所有相关的引用变量都不再缓。如果把 volatile 移到 Test 里面的isRunnning 变量上去,程序也能正常结束,说明JVM对所有指定了变量的

public class Test {
    private volatile boolean isRunnning = true;

    public static void main(String[] args) throws Exception {
        PrintString printString = new PrintString();
        Thread thread = new Thread(printString,"Thread-A");
        thread.start();
        Thread.sleep(1000);
        System.out.println(" 我要停止它!" + Thread.currentThread().getName());
        Test test = new Test();
        test.setRunnning(false);
        printString.setTest(test);
    }

    public boolean isRunnning() {
        return isRunnning;
    }

    public void setRunnning(boolean runnning) {
        isRunnning = runnning;
    }
}

class PrintString implements Runnable{
    private Test test = new Test();

    @Override
    public void run() {
        System.out.println("Thread begin: "+Thread.currentThread().getName());
        while (test.isRunnning()){
        }
        System.out.println("Thread end: "+Thread.currentThread().getName());
    }

    public void setRunnning(boolean runnning) {
        test.setRunnning(runnning);
    }

    public void setTest(Test test) {
        this.test = test;
    }
}

虽然test 没有用 volatile 修饰,程序也能正常结束,去掉isRunnning 的volatile ,会出现死循环,说明JVM去拿一个 volatile 的内部变量时,不管外部变量是否用 volatile 修饰,都会去主存中找到最新的外部变量,然后再去找最新的内部变量。

总之为了安全考虑,只要是可能会多线程同时读写的变量,最好都加上 volatile。

参考文章:

Memory footprint of the JVM(转至springteam)

The JVM can be a complex beast. Thankfully, much of that complexity is under the hood, and we as application developers and deployers often don’t have to worry about it too much. With the rise of container-based deployment strategies, one area of complexity that needs some attention is the JVM’s memory footprint.

Two kinds of memory

The JVM divides its memory into two main categories: heap memory and non-heap memory. Heap memory is the part with which people are typically the most familiar. It’s where objects that are created by the application are stored. They remain there until they are no longer referenced and are garbage collected. Typically, the amount of heap that an application is using will fluctuate as a function of the current load.

The JVM’s non-heap memory is divided into several different areas. We can use the HotSpot VM’s native memory tracking (NMT) to examine its memory usage across these areas. Note that, while NMT does not track all native memory usage (it does not track third party native code memory allocations, for example), it is sufficient for a large class of typical Spring applications. NMT can be used by starting the application with -XX:NativeMemoryTracking=summary and then using jcmd <pid> VM.native_memory summary to display the memory usage summary.

Let’s illustrate the use of NMT by looking at an application, in this case our old friend, Petclinic. The following pie chart shows the JVM’s memory usage as reported by NMT (minus its own overhead) when starting Petclinic with a 48MB max heap (-Xmx48M):

As you can see non-heap memory accounts for the vast majority of the JVM’s memory usage with the heap memory accounting for only one sixth of the total. In this case it was roughly 44MB (with 33MB of that being used immediately after garbage collection). The non-heap memory usage was 223MB in total.

Native Memory areas

  • Compressed class space: used to store information about the classes that have been loaded. Constrained by MaxMetaspaceSize. A function of the number of classes that have been loaded.
  • Thread: memory used by threads in the JVM. A function of the number of threads that are running.
  • Code cache: memory used by the JIT to store its output. A function of the number of classes that have been loaded. Constrained by ReservedCodeCacheSize. Can be reduced by tuning the JIT to, for example, disable tiered compilation.
  • GC: stores data used by the GC. Varies depending on which garbage collector is being used.
  • Symbol: stores symbols such as field names, method signatures, and interned strings. Excessive symbol memory usage can be an indicator that Strings have been interned too aggressively.
  • Internal: stores other internal data that does not fit into any of the other areas.

Differences

Compared to heap memory, non-heap memory is less likely to vary under load. Once an application has loaded all of the classes that it will use and the JIT is fully warmed up, things will settle into a steady state. To see a reduction in compressed class space usage, the class loader that loaded the classes needs to be garbage collected. This was more common in the past when applications were deployed to servlet containers or app servers – the application’s class loader would be garbage collected when the application was undeployed – but rarely happens with modern approaches to application deployment.

Sizing the JVM

Configuring the JVM to make efficient use of a given amount of available RAM isn’t easy. If you launch the JVM with -Xmx16M and expect it to use, at most, 16MB of RAM you are in for a nasty surprise.

An interesting area when sizing the JVM is the JIT’s code cache. By default, the HotSpot JVM will use up to 240MB. If the code cache is too small the JIT will run out of space to store its output and performance will suffer as a result. If the cache is too large, memory may be wasted. When sizing the code cache, it’s important to look at the effect on both your application’s memory usage and its performance.

When running in a Docker container, recent versions of Java are now aware of the container’s memory limits and attempt to size the JVM accordingly. Unfortunately, this sizing often over-allocates non-heap memory and under-allocates the heap. Say you have an application running in a container with 2 CPUs and 512MB of memory available. You want it to be able to handle more load so you double the CPUs to 4 and the memory to 1GB. As we discussed above, heap usage typically varies depending on the load, and non-heap usage much less so. Therefore, we’d like the vast majority of the extra 512MB of memory to be given to the heap to cope with the increased load. Unfortunately, the JVM does not do so by default and will allocate the additional memory more equally between its heap and non-heap areas.

Thankfully, the CloudFoundry team have a wealth of knowledge about the JVM’s memory footprint. If you’re pushing apps to CloudFoundry, the build pack will automatically apply this knowledge for you. If you’re not using CloudFoudry, or you’d like to understand more about how to size your JVM, the design document for version three of the Java buildpack’s memory calculator provides some highly recommended further reading.

以下摘自stackoverflow

Virtual memory used by a Java process extends far beyond just Java Heap. You know, JVM includes many subsytems: Garbage Collector, Class Loading, JIT compilers etc., and all these subsystems require certain amount of RAM to function.

JVM is not the only consumer of RAM. Native libraries (including standard Java Class Library) may also allocate native memory. And this won’t be even visible to Native Memory Tracking. Java application itself can also use off-heap memory by means of direct ByteBuffers.

So what takes memory in a Java process?

JVM parts (mostly shown by Native Memory Tracking)

  1. Java HeapThe most obvious part. This is where Java objects live. Heap takes up to -Xmx amount of memory.
  2. Garbage CollectorGC structures and algorithms require additional memory for heap management. These structures are Mark Bitmap, Mark Stack (for traversing object graph), Remembered Sets (for recording inter-region references) and others. Some of them are directly tunable, e.g. -XX:MarkStackSizeMax, others depend on heap layout, e.g. the larger are G1 regions (-XX:G1HeapRegionSize), the smaller are remembered sets.GC memory overhead varies between GC algorithms. -XX:+UseSerialGC and -XX:+UseShenandoahGC have the smallest overhead. G1 or CMS may easily use around 10% of total heap size.
  3. Code CacheContains dynamically generated code: JIT-compiled methods, interpreter and run-time stubs. Its size is limited by -XX:ReservedCodeCacheSize (240M by default). Turn off -XX:-TieredCompilation to reduce the amount of compiled code and thus the Code Cache usage.
  4. CompilerJIT compiler itself also requires memory to do its job. This can be reduced again by switching off Tiered Compilation or by reducing the number of compiler threads: -XX:CICompilerCount.
  5. Class loadingClass metadata (method bytecodes, symbols, constant pools, annotations etc.) is stored in off-heap area called Metaspace. The more classes are loaded – the more metaspace is used. Total usage can be limited by -XX:MaxMetaspaceSize (unlimited by default) and -XX:CompressedClassSpaceSize (1G by default).
  6. Symbol tablesTwo main hashtables of the JVM: the Symbol table contains names, signatures, identifiers etc. and the String table contains references to interned strings. If Native Memory Tracking indicates significant memory usage by a String table, it probably means the application excessively calls String.intern.
  7. ThreadsThread stacks are also responsible for taking RAM. The stack size is controlled by -Xss. The default is 1M per thread, but fortunately the things are not so bad. OS allocates memory pages lazily, i.e. on the first use, so the actual memory usage will be much lower (typically 80-200 KB per thread stack). I wrote a script to estimate how much of RSS belongs to Java thread stacks.There are other JVM parts that allocate native memory, but they do not usually play a big role in total memory consumption.

Direct buffers

An application may explicitly request off-heap memory by calling ByteBuffer.allocateDirect. The default off-heap limit is equal to -Xmx, but it can be overridden with -XX:MaxDirectMemorySize. Direct ByteBuffers are included in Other section of NMT output (or Internal before JDK 11).

The amount of used direct memory is visible through JMX, e.g. in JConsole or Java Mission Control:

BufferPool MBean

Besides direct ByteBuffers there can be MappedByteBuffers – the files mapped to virtual memory of a process. NMT does not track them, however, MappedByteBuffers can also take physical memory. And there is no a simple way to limit how much they can take. You can just see the actual usage by looking at process memory map: pmap -x <pid>

Address           Kbytes    RSS    Dirty Mode  Mapping
...
00007f2b3e557000   39592   32956       0 r--s- some-file-17405-Index.db
00007f2b40c01000   39600   33092       0 r--s- some-file-17404-Index.db
                           ^^^^^               ^^^^^^^^^^^^^^^^^^^^^^^^

Native libraries

JNI code loaded by System.loadLibrary can allocate as much off-heap memory as it wants with no control from JVM side. This also concerns standard Java Class Library. In particular, unclosed Java resources may become a source of native memory leak. Typical examples are ZipInputStream or DirectoryStream.

JVMTI agents, in particular, jdwp debugging agent – can also cause excessive memory consumption.

This answer describes how to profile native memory allocations with async-profiler.

Allocator issues

A process typically requests native memory either directly from OS (by mmap system call) or by using malloc – standard libc allocator. In turn, malloc requests big chunks of memory from OS using mmap, and then manages these chunks according to its own allocation algorithm. The problem is – this algorithm can lead to fragmentation and excessive virtual memory usage.

jemalloc, an alternative allocator, often appears smarter than regular libc malloc, so switching to jemalloc may result in a smaller footprint for free.

Conclusion

There is no guaranteed way to estimate full memory usage of a Java process, because there are too many factors to consider.

Total memory = Heap + Code Cache + Metaspace + Symbol tables +
               Other JVM structures + Thread stacks +
               Direct buffers + Mapped files +
               Native Libraries + Malloc overhead + ...

It is possible to shrink or limit certain memory areas (like Code Cache) by JVM flags, but many others are out of JVM control at all.

One possible approach to setting Docker limits would be to watch the actual memory usage in a “normal” state of the process. There are tools and techniques for investigating issues with Java memory consumption: Native Memory Trackingpmapjemallocasync-profiler.

SpringBoot多模块项目在IDEA下运行调试

一个SpringBoot多模块项目的工程目录如下:

myProject
--api
--ops
--share

现在要在IDEA中调试运行ops模块,程序启动后,所有页面无法访问,提示:

WARN  |-o.e.j.server.handler.ErrorHandler:106  - Error page loop /WEB-INF/jsp/common/404.jsp

这里提示错误页面循环,是因为程序找不到要渲染的jsp页面时,设置了跳转到404页面,结果404页面也找不到了,就出现了错误循环。Jetty查找JSP的文件路径为:

//WebAppContext.getResource(String path):
_baseResource + jsp.prefix + jsp.fileName + jsp.suffix
其中_baseResource 的路径查找步骤为:
//DocumentRoot.getValidDirectory()
getWarFileDocumentRoot();
getExplodedWarFileDocumentRoot();
getCommonDocumentRoot();//判断当前运行目录下是否存在src/main/webapp | public | static 这3个目录中的一个,如果存在,则将当前目录设置为_baseResource
如果根据上面的规则都找不到合适的DocumentRoot,则会使用创建一个运行时的临时目录,项目的JSP文件在临时目录里面肯定是找不到的,所以就不能正常显示了。

解决方案:
在IDEA的Run/Debug Configurations中,将Working directory设置为要运行的模块目录绝对路径,例如/opt/myProject/ops。

找不到JSP的问题算是解决了,如果项目中有用到freemarker和taglib,还会面临一个问题,

Caused by: freemarker.template.TemplateModelException: Error while loading tag library for URI “/my-taglib” from TLD location “servletContext:/my-taglib”; see cause exception.

freemarker查找taglib时,默认使用TaglibFactory.DEFAULT_META_INF_TLD_SOURCES  = Collections.singletonList(WebInfPerLibJarMetaInfTldSource.INSTANCE)
查找tld文件的路径为:sevletContext:/WEB-INF/lib/*.{jar,zip}/META-INF/**/*.tld
在IDEA下直接run SpringBoot,相关的依赖包并不会拷贝到/WEB-INF/lib/下面去,所以就查找不到这些jar包里面的tld文件了,需要让freemarker去classpath中的所有jar包里面去查找,解决方案:

public class IdeFreeMarkerConfigurer extends FreeMarkerConfigurer {

    @Override
    public void afterPropertiesSet() throws IOException, TemplateException {
        super.afterPropertiesSet();          

        super.getTaglibFactory().setMetaInfTldSources(Lists.newArrayList(
                TaglibFactory.WebInfPerLibJarMetaInfTldSource.INSTANCE,
                new TaglibFactory.ClasspathMetaInfTldSource(Pattern.compile(".*\\.jar$", Pattern.DOTALL))));
    }
}

在XML或者Java代码中,配置使用这个FreeMarkerConfigurer 就可以了

 

 

你真的会写单例模式吗——Java实现

饿汉法

顾名思义,饿汉法就是在第一次引用该类的时候就创建对象实例,而不管实际是否需要创建。代码如下:

public class Singleton {  
    private static Singleton = new Singleton();
    private Singleton() {}
    public static getSignleton(){
        return singleton;
    }
}

这样做的好处是编写简单,但是无法做到延迟创建对象。但是我们很多时候都希望对象可以尽可能地延迟加载,从而减小负载,所以就需要下面的懒汉法:

单线程写法

这种写法是最简单的,由私有构造器和一个公有静态工厂方法构成,在工厂方法中对singleton进行null判断,如果是null就new一个出来,最后返回singleton对象。这种方法可以实现延时加载,但是有一个致命弱点:线程不安全。如果有两条线程同时调用getSingleton()方法,就有很大可能导致重复创建对象。

public class Singleton {
    private static Singleton singleton = null;
    private Singleton(){}
    public static Singleton getSingleton() {
        if(singleton == null) singleton = new Singleton();
        return singleton;
    }
}

考虑线程安全的写法

这种写法考虑了线程安全,将对singleton的null判断以及new的部分使用synchronized进行加锁。同时,对singleton对象使用volatile关键字进行限制,保证其对所有线程的可见性,并且禁止对其进行指令重排序优化。如此即可从语义上保证这种单例模式写法是线程安全的。注意,这里说的是语义上,实际使用中还是存在小坑的,会在后文写到。

public class Singleton {
    private static volatile Singleton singleton = null;
 
    private Singleton(){}
 
    public static Singleton getSingleton(){
        synchronized (Singleton.class){
            if(singleton == null){
                singleton = new Singleton();
            }
        }
        return singleton;
    }   
}

兼顾线程安全和效率的写法

虽然上面这种写法是可以正确运行的,但是其效率低下,还是无法实际应用。因为每次调用getSingleton()方法,都必须在synchronized这里进行排队,而真正遇到需要new的情况是非常少的。所以,就诞生了第三种写法:

public class Singleton {
    private static volatile Singleton singleton = null;
 
    private Singleton(){}
 
    public static Singleton getSingleton(){
        if(singleton == null){
            synchronized (Singleton.class){
                if(singleton == null){
                    singleton = new Singleton();
                }
            }
        }
        return singleton;
    }   
}

这种写法被称为“双重检查锁”,顾名思义,就是在getSingleton()方法中,进行两次null检查。看似多此一举,但实际上却极大提升了并发度,进而提升了性能。为什么可以提高并发度呢?就像上文说的,在单例中new的情况非常少,绝大多数都是可以并行的读操作。因此在加锁前多进行一次null检查就可以减少绝大多数的加锁操作,执行效率提高的目的也就达到了。

那么,这种写法是不是绝对安全呢?前面说了,从语义角度来看,并没有什么问题。但是其实还是有坑。说这个坑之前我们要先来看看volatile这个关键字。其实这个关键字有两层语义。第一层语义相信大家都比较熟悉,就是可见性。可见性指的是在一个线程中对该变量的修改会马上由工作内存(Work Memory)写回主内存(Main Memory),所以会马上反应在其它线程的读取操作中。顺便一提,工作内存和主内存可以近似理解为实际电脑中的高速缓存和主存,工作内存是线程独享的,主存是线程共享的。volatile的第二层语义是禁止指令重排序优化。大家知道我们写的代码(尤其是多线程代码),由于编译器优化,在实际执行的时候可能与我们编写的顺序不同。编译器只保证程序执行结果与源代码相同,却不保证实际指令的顺序与源代码相同。这在单线程看起来没什么问题,然而一旦引入多线程,这种乱序就可能导致严重问题。volatile关键字就可以从语义上解决这个问题。

例如,考虑下面的事件序列:

  1. 线程A发现变量没有被初始化, 然后它获取锁并开始变量的初始化。
  2. 由于某些编程语言的语义,编译器生成的代码允许在线程A执行完变量的初始化之前,更新变量并将其指向部分初始化的对象。
  3. 线程B发现共享变量已经被初始化,并返回变量。由于线程B确信变量已被初始化,它没有获取锁。如果在A完成初始化之前共享变量对B可见(这是由于A没有完成初始化或者因为一些初始化的值还没有穿过B使用的内存(缓存一致性)),程序很可能会崩溃。

Symantec JIT 编译 singletons[i].reference = new Singleton(); 这段代码时,如果不加volatile关键词,会生成如下字节码:

0206106A   mov         eax,0F97E78h
0206106F   call        01F6B210                  ; allocate space for
                                                 ; Singleton, return result in eax
02061074   mov         dword ptr [ebp],eax       ; EBP is &singletons[i].reference 
                                                ; store the unconstructed object here.
02061077   mov         ecx,dword ptr [eax]       ; dereference the handle to
                                                 ; get the raw pointer
02061079   mov         dword ptr [ecx],100h      ; Next 4 lines are
0206107F   mov         dword ptr [ecx+4],200h    ; Singleton's inlined constructor
02061086   mov         dword ptr [ecx+8],400h
0206108D   mov         dword ptr [ecx+0Ch],0F84030h

可以看到,在执行Singleton的构造函数之前,Singleton的新实例就被赋值给了singletons[i].reference,这在Java内存模型中是完全合法的。

注意,前面反复提到“从语义上讲是没有问题的”,但是很不幸,禁止指令重排优化这条语义直到jdk1.5以后才能正确工作。此前的JDK中即使将变量声明为volatile也无法完全避免重排序所导致的问题。所以,在jdk1.5版本前,双重检查锁形式的单例模式是无法保证线程安全的。

静态内部类法

那么,有没有一种延时加载,并且能保证线程安全的简单写法呢?我们可以把Singleton实例放到一个静态内部类中,这样就避免了静态实例在Singleton类加载的时候就创建对象,并且由于静态内部类只会被加载一次,所以这种写法也是线程安全的:

public class Singleton {
    private static class Holder {
        private static Singleton singleton = new Singleton();
    }
 
    private Singleton(){}
 
    public static Singleton getSingleton(){
        return Holder.singleton;
    }
}

但是,上面提到的所有实现方式都有两个共同的缺点:

  • 都需要额外的工作(Serializable、transient、readResolve())来实现序列化,否则每次反序列化一个序列化的对象实例时都会创建一个新的实例。
  • 可能会有人使用反射强行调用我们的私有构造器(如果要避免这种情况,可以修改构造器,让它在创建第二个实例的时候抛异常)。

枚举写法

当然,还有一种更加优雅的方法来实现单例模式,那就是枚举写法:

public enum Singleton {
    INSTANCE;
    private String name;
    public String getName(){
        return name;
    }
    public void setName(String name){
        this.name = name;
    }
}

使用枚举除了线程安全和防止反射强行调用构造器之外,还提供了自动序列化机制,防止反序列化的时候创建新的对象。因此,Effective Java推荐尽可能地使用枚举来实现单例。

总结

代码没有一劳永逸的写法,只有在特定条件下最合适的写法。在不同的平台、不同的开发环境(尤其是jdk版本)下,自然有不同的最优解(或者说较优解)。
比如枚举,虽然Effective Java中推荐使用,但是在Android平台上却是不被推荐的。在这篇Android Training中明确指出:

Enums often require more than twice as much memory as static constants. You should strictly avoid using enums on Android.

再比如双重检查锁法,不能在jdk1.5之前使用,而在Android平台上使用就比较放心了(一般Android都是jdk1.6以上了,不仅修正了volatile的语义问题,还加入了不少锁优化,使得多线程同步的开销降低不少)。

最后,不管采取何种方案,请时刻牢记单例的三大要点:

  • 线程安全
  • 延迟加载
  • 序列化与反序列化安全

参考资料

《Effective Java(第二版)》
《深入理解Java虚拟机——JVM高级特性与最佳实践(第二版)》
双重检查锁定模式
The “Double-Checked Locking is Broken” Declaration

为什么不能在for循环中remove掉ArrayList中的对象

先看一段错误的代码,执行的时候不会报错,但是结果和编码人的预期却是不一样的:

List<String> a = new ArrayList<>();
a.add("1");
a.add("2");
for (String temp : a) {
    System.out.println(temp);
    if("1".equals(temp)){
        a.remove(temp);
    }
}

再看一段错误的代码,执行的时候会直接抛出java.util.ConcurrentModificationException:

List<String> a = new ArrayList<>();
a.add("1");
a.add("2");
for (String temp : a) {
    System.out.println(temp);
    if("2".equals(temp)){
        a.remove(temp);
    }
}

奇怪了,为什么第一段代码不会抛出异常,而是执行一次循环就结束了呢,使用javap查看编译好的class文件:

26: aload_1
27: invokeinterface #7,  1            // InterfaceMethod java/util/List.iterator:()Ljava/util/Iterator;
32: astore_2
33: aload_2
34: invokeinterface #8,  1            // InterfaceMethod java/util/Iterator.hasNext:()Z
39: ifeq          72
42: aload_2
43: invokeinterface #9,  1            // InterfaceMethod java/util/Iterator.next:()Ljava/lang/Object;
48: checkcast     #10                 // class java/lang/String
51: astore_3

在执行for循环时,其实是先调用了ArrayList中的iterator方法得到一个Iterator,然后每次循环时,判断Iterator.hasNext,如果hasNext(根据cursor != size判断),则调用Iterator.next,此时就会判断List是否被意外修改(checkForComodification),if (modCount != expectedModCount)则抛出ConcurrentModificationException。如果remove的cursor刚好为size-2,即倒数第二个对象,下次判断hasNext时,cursor == size了,就返回false,认为没有后面的元素了,不会再调用Iterator.next,因此也不会抛出异常,但是也不会把最后一个没有遍历过的元素拿出来进行业务处理。

查看Iterator.remove()的源码,处理基本和ArrayList.remove一致,比较明显的区别就是,Iterator.remove完成后,会调整当前的游标值,继续遍历时就不会导致遍历数据错误,而ArrayList.remove不会调整游标值,如果继续遍历,就会漏掉数据而出错,因此需要抛出ConcurrentModificationException避免被错误使用。