亚欧色一区w666天堂,色情一区二区三区免费看,少妇特黄A片一区二区三区,亚洲人成网站999久久久综合,国产av熟女一区二区三区

  • 發布文章
  • 消息中心
點贊
收藏
評論
分享
原創

記一次Java full gc的分析過程

2023-05-04 01:22:25
24
0

有一次在開發環境做一個接口的壓測時,結果很不理想。于是用命令查了一下進程的gc情況,每500毫秒查一次,連續查100次

jstat -gc 2847106 500 100

結果發現Full gc次數比Young gc次數多得多,高達1670次,平均每幾秒就一次Full gc。

如此頻繁的Full gc,可以看出每次Full gc之后都沒能清掉堆內的對象釋放內存,因此需要打印jvm內存中對象數量來看下是什么對象常駐內存沒有釋放。執行

 jmap -histo 2847106 | less

結果可以看出是groovy相關的對象占用了大部分的堆空間,問題是工程中并沒有直接使用groovy,需要找出是哪個jar間接使用了groovy

原來是工程中使用了json-path來解釋接口請求中的json入參(json-path可以使用搜索路徑來直接讀取json中的某個字段的值),而json-path使用了groovy腳本來讀取json數據。

// json-path 將搜索路徑都轉成了 JSONAssertion
class JSONAssertion implements Assertion {
  String key;
  Map<String, Object> params;

  def Object getResult(object, config) {
    Object result = getAsJsonObject(object)
    return result;
  }

  def getAsJsonObject(object) {
    key = escapePath(key, hyphen(), attributeGetter(), integer(), properties(), classKeyword());
    def result;
    if (key == "\$" || key == "") {
      result = object
    } else {
      def root = 'restAssuredJsonRootObject'
      try {
        def expr;
        if (key =~ /^\[\d+\].*/) {
          expr = "$root$key"
        } else {
          expr = "$root.$key"
        }
        result = 
      } catch (MissingPropertyException e) {
        // This means that a param was used that was not defined
        String error = String.format("The parameter \"%s\" was used but not defined. Define parameters using the JsonPath.params(...) function", e.property);
        throw new IllegalArgumentException(error, e);
      } catch (Exception e) {
        String error = e.getMessage().replace("startup failed:","Invalid JSON expression:").replace("$root.", generateWhitespace(root.length()));
        throw new IllegalArgumentException(error, e);
      }
    }
    return result
  }

  def String description() {
    return "JSON path"
  }

  private def  {
      Map<String, Object> newParams;
      // Create parameters from given ones
      if(params!=null) {
          newParams=new HashMap<>(params);
      } else {
          newParams=new HashMap<>();
      }
      // Add object to evaluate
      newParams.put(root, object);
      // Create shell with variables set
      GroovyShell sh = new GroovyShell(new Binding(newParams));
      // Run
      return sh.evaluate(expr);
  }
}

那為什么使用groovy會有這樣的問題呢?

每次groovy都會根據腳本內容生成一個class對象和生成一個新的classloader,由這個classloader去加載class

    public GroovyShell(ClassLoader parent, Binding binding, final CompilerConfiguration config) {
        if (binding == null) {
            throw new IllegalArgumentException("Binding must not be null.");
        }
        if (config == null) {
            throw new IllegalArgumentException("Compiler configuration must not be null.");
        }
        final ClassLoader parentLoader = (parent!=null)?parent:GroovyShell.class.getClassLoader();
        this.loader = AccessController.doPrivileged(new PrivilegedAction<GroovyClassLoader>() {
            public GroovyClassLoader run() {
                return new GroovyClassLoader(parentLoader,config);
            }
        });
        this.context = binding;        
        this.config = config;
    }

繼續查看GroovyClassLoader的源碼,發現有兩個用于緩存的map,sourceCache用來緩存groovy腳本文件名,classCache用來緩存已編譯的class。從源碼中看出,腳本沒有緩存到sourceCache,但會緩存到classCache,導致class對象沒有及時被full gc回收。

    /**
     * this cache contains the loaded classes or PARSING, if the class is currently parsed
     */
    protected final Map<String, Class> classCache = new HashMap<String, Class>();

    /**
     * This cache contains mappings of file name to class. It is used
     * to bypass compilation.
     */
    protected final Map<String, Class> sourceCache = new HashMap<String, Class>();
    /**
     * Parses the groovy code contained in codeSource and returns a java class.
     */
    private Class parseClass(final GroovyCodeSource codeSource) throws CompilationFailedException {
        // Don't cache scripts
        return loader.parseClass(codeSource, false);
    }
    /**
     * Parses the given code source into a Java class. If there is a class file
     * for the given code source, then no parsing is done, instead the cached class is returned.
     *
     * @param shouldCacheSource if true then the generated class will be stored in the source cache
     * @return the main class defined in the given script
     */
    public Class parseClass(GroovyCodeSource codeSource, boolean shouldCacheSource) throws CompilationFailedException {
        synchronized (sourceCache) {
            Class answer = sourceCache.get(codeSource.getName());
            if (answer != null) return answer;
            answer = doParseClass(codeSource);
            if (shouldCacheSource) sourceCache.put(codeSource.getName(), answer);
            return answer;
        }
    }

    private Class doParseClass(GroovyCodeSource codeSource) {
        validate(codeSource);
        Class answer;  // Was neither already loaded nor compiling, so compile and add to cache.
        CompilationUnit unit = createCompilationUnit(config, codeSource.getCodeSource());
        if (recompile!=null && recompile || recompile==null && config.getRecompileGroovySource()) {
            unit.addFirstPhaseOperation(TimestampAdder.INSTANCE, CompilePhase.CLASS_GENERATION.getPhaseNumber());
        }
        SourceUnit su = null;
        File file = codeSource.getFile();
        if (file != null) {
            su = unit.addSource(file);
        } else {
            URL url = codeSource.getURL();
            if (url != null) {
                su = unit.addSource(url);
            } else {
                su = unit.addSource(codeSource.getName(), codeSource.getScriptText());
            }
        }

        ClassCollector collector = createCollector(unit, su);
        unit.setClassgenCallback(collector);
        int goalPhase = Phases.CLASS_GENERATION;
        if (config != null && config.getTargetDirectory() != null) goalPhase = Phases.OUTPUT;
        unit.compile(goalPhase);

        answer = collector.generatedClass;
        String mainClass = su.getAST().getMainClassName();
        for (Object o : collector.getLoadedClasses()) {
            Class clazz = (Class) o;
            String clazzName = clazz.getName();
            definePackageInternal(clazzName);
            setClassCacheEntry(clazz);
            if (clazzName.equals(mainClass)) answer = clazz;
        }
        return answer;
    }

找到原因后,項目中就將json-path去掉了。因為json的搜索路徑是固定的,之前用json-path就是想少寫點代碼,現在知道有性能問題只能去掉了。

去掉json-path之后就沒有頻繁的Full gc了。

0條評論
0 / 1000
白龍馬
14文章數
0粉絲數
白龍馬
14 文章 | 0 粉絲
原創

記一次Java full gc的分析過程

2023-05-04 01:22:25
24
0

有一次在開發環境做一個接口的壓測時,結果很不理想。于是用命令查了一下進程的gc情況,每500毫秒查一次,連續查100次

jstat -gc 2847106 500 100

結果發現Full gc次數比Young gc次數多得多,高達1670次,平均每幾秒就一次Full gc。

如此頻繁的Full gc,可以看出每次Full gc之后都沒能清掉堆內的對象釋放內存,因此需要打印jvm內存中對象數量來看下是什么對象常駐內存沒有釋放。執行

 jmap -histo 2847106 | less

結果可以看出是groovy相關的對象占用了大部分的堆空間,問題是工程中并沒有直接使用groovy,需要找出是哪個jar間接使用了groovy

原來是工程中使用了json-path來解釋接口請求中的json入參(json-path可以使用搜索路徑來直接讀取json中的某個字段的值),而json-path使用了groovy腳本來讀取json數據。

// json-path 將搜索路徑都轉成了 JSONAssertion
class JSONAssertion implements Assertion {
  String key;
  Map<String, Object> params;

  def Object getResult(object, config) {
    Object result = getAsJsonObject(object)
    return result;
  }

  def getAsJsonObject(object) {
    key = escapePath(key, hyphen(), attributeGetter(), integer(), properties(), classKeyword());
    def result;
    if (key == "\$" || key == "") {
      result = object
    } else {
      def root = 'restAssuredJsonRootObject'
      try {
        def expr;
        if (key =~ /^\[\d+\].*/) {
          expr = "$root$key"
        } else {
          expr = "$root.$key"
        }
        result = 
      } catch (MissingPropertyException e) {
        // This means that a param was used that was not defined
        String error = String.format("The parameter \"%s\" was used but not defined. Define parameters using the JsonPath.params(...) function", e.property);
        throw new IllegalArgumentException(error, e);
      } catch (Exception e) {
        String error = e.getMessage().replace("startup failed:","Invalid JSON expression:").replace("$root.", generateWhitespace(root.length()));
        throw new IllegalArgumentException(error, e);
      }
    }
    return result
  }

  def String description() {
    return "JSON path"
  }

  private def  {
      Map<String, Object> newParams;
      // Create parameters from given ones
      if(params!=null) {
          newParams=new HashMap<>(params);
      } else {
          newParams=new HashMap<>();
      }
      // Add object to evaluate
      newParams.put(root, object);
      // Create shell with variables set
      GroovyShell sh = new GroovyShell(new Binding(newParams));
      // Run
      return sh.evaluate(expr);
  }
}

那為什么使用groovy會有這樣的問題呢?

每次groovy都會根據腳本內容生成一個class對象和生成一個新的classloader,由這個classloader去加載class

    public GroovyShell(ClassLoader parent, Binding binding, final CompilerConfiguration config) {
        if (binding == null) {
            throw new IllegalArgumentException("Binding must not be null.");
        }
        if (config == null) {
            throw new IllegalArgumentException("Compiler configuration must not be null.");
        }
        final ClassLoader parentLoader = (parent!=null)?parent:GroovyShell.class.getClassLoader();
        this.loader = AccessController.doPrivileged(new PrivilegedAction<GroovyClassLoader>() {
            public GroovyClassLoader run() {
                return new GroovyClassLoader(parentLoader,config);
            }
        });
        this.context = binding;        
        this.config = config;
    }

繼續查看GroovyClassLoader的源碼,發現有兩個用于緩存的map,sourceCache用來緩存groovy腳本文件名,classCache用來緩存已編譯的class。從源碼中看出,腳本沒有緩存到sourceCache,但會緩存到classCache,導致class對象沒有及時被full gc回收。

    /**
     * this cache contains the loaded classes or PARSING, if the class is currently parsed
     */
    protected final Map<String, Class> classCache = new HashMap<String, Class>();

    /**
     * This cache contains mappings of file name to class. It is used
     * to bypass compilation.
     */
    protected final Map<String, Class> sourceCache = new HashMap<String, Class>();
    /**
     * Parses the groovy code contained in codeSource and returns a java class.
     */
    private Class parseClass(final GroovyCodeSource codeSource) throws CompilationFailedException {
        // Don't cache scripts
        return loader.parseClass(codeSource, false);
    }
    /**
     * Parses the given code source into a Java class. If there is a class file
     * for the given code source, then no parsing is done, instead the cached class is returned.
     *
     * @param shouldCacheSource if true then the generated class will be stored in the source cache
     * @return the main class defined in the given script
     */
    public Class parseClass(GroovyCodeSource codeSource, boolean shouldCacheSource) throws CompilationFailedException {
        synchronized (sourceCache) {
            Class answer = sourceCache.get(codeSource.getName());
            if (answer != null) return answer;
            answer = doParseClass(codeSource);
            if (shouldCacheSource) sourceCache.put(codeSource.getName(), answer);
            return answer;
        }
    }

    private Class doParseClass(GroovyCodeSource codeSource) {
        validate(codeSource);
        Class answer;  // Was neither already loaded nor compiling, so compile and add to cache.
        CompilationUnit unit = createCompilationUnit(config, codeSource.getCodeSource());
        if (recompile!=null && recompile || recompile==null && config.getRecompileGroovySource()) {
            unit.addFirstPhaseOperation(TimestampAdder.INSTANCE, CompilePhase.CLASS_GENERATION.getPhaseNumber());
        }
        SourceUnit su = null;
        File file = codeSource.getFile();
        if (file != null) {
            su = unit.addSource(file);
        } else {
            URL url = codeSource.getURL();
            if (url != null) {
                su = unit.addSource(url);
            } else {
                su = unit.addSource(codeSource.getName(), codeSource.getScriptText());
            }
        }

        ClassCollector collector = createCollector(unit, su);
        unit.setClassgenCallback(collector);
        int goalPhase = Phases.CLASS_GENERATION;
        if (config != null && config.getTargetDirectory() != null) goalPhase = Phases.OUTPUT;
        unit.compile(goalPhase);

        answer = collector.generatedClass;
        String mainClass = su.getAST().getMainClassName();
        for (Object o : collector.getLoadedClasses()) {
            Class clazz = (Class) o;
            String clazzName = clazz.getName();
            definePackageInternal(clazzName);
            setClassCacheEntry(clazz);
            if (clazzName.equals(mainClass)) answer = clazz;
        }
        return answer;
    }

找到原因后,項目中就將json-path去掉了。因為json的搜索路徑是固定的,之前用json-path就是想少寫點代碼,現在知道有性能問題只能去掉了。

去掉json-path之后就沒有頻繁的Full gc了。

文章來自個人專欄
文章 | 訂閱
0條評論
0 / 1000
請輸入你的評論
0
0