Friday, 10 February 2017

Java native library memory profiling using Valgrind

Our Java application  interacts lot with our native libraries for various computation purposes, for those native we wanted to conduct the memory profile using “Valgrind”.
When I first started JVM under Valgrind which never took off because of memory usage and huge code instrumentation was needed to done by Valgrind.


Finally I had to serialize all JNI interactions using java.lang.reflect.InvocationHandler during the Java application run and later on replayed JNI interactions under Valgrind environment to get the memory profile of native libraries.


Following the diagram describing our approach:



Note: It is an invasive approach (I had to change application code for recording purpose) because of the use of java.lang.reflect.InvocationHandler for recording, however Spring AOP could also be used in cases where non-invasive approach is needed.
For recording I used JBOSS COT to avoid implementing serializable interface within all our model classes, alternatively serializable interface can be implemented within all classes.

Code snipped of recorder:
public class JNIInteractionRecorder implements InvocationHandler {
  static public String OUTPUT_FILE = "jni_interactions.output";

   // Application JNI wrapper which needs to be recorded.
    private final JNIWrapper OriginalWrapper;

    /**
     * output stream parameter; used JBOSS because then we do not need to have
    * serializable interface implemented.
    */
    private JBossObjectOutputStream output;

       /**
         * Input stream parameter; used JBOSS library because then we do not need to have
         * serializable interface implemented.
       */
    Private JBossObjectInputStream input

 JNIInteractionRecorder() {
   OutputStream outFile = new FileOutputStream(dirName  +  "/"   + OUTPUT_FILE);
            OutputStream buffer = new BufferedOutputStream( outFile );
            output = new JBossObjectOutputStream( buffer );
}

@Override
    protected void finalize() throws Throwable {
               output.flush();
        output.close();
        super.finalize();
    }

@Override
    public synchronized Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
               try {
                              Object result = method.invoke(wrapper, args);
                              output.writeUTF(method.getName());
                              output.writeObject(args);
                              output.flush();

                              if (currentDataLen > MAX_FILE_SIZE) {
                                             rollover();
                              }
                              return result;
               }
               catch (InvocationTargetException e) {
                              e.printStackTrace();
                              throw e.getCause();
               }
    }
}


/**
     * Replay the test under Valgrind environment.
     */
    public void runIt() {
        try {
            while (input.available() > 0) {
                methodName = input.readUTF();
                array = (Object[])input.readObject();

                Method method = null;
                try {
                    method = getWrapperType().getMethod(methodName, convertToClassArray(array));
                }
                catch (NoSuchMethodException e) {
                                             e.printStackTrace();
                }

                if (method == null) {
                    // try with the collection class
                    try {
                        method = getWrapperType().getMethod(methodName, convertToClassArrayWithCollecion(array));
                    }
                    catch (NoSuchMethodException e) {
                        e.printStackTrace();
                    }
                }

                if (method != null) {

                    try {
                        method.invoke(wrapper, array);
                    }
                    catch (InvocationTargetException e) {
                        System.out.println("cause:"  e.getCause().getMessage());
                        e.printStackTrace();
                    }
                }
            }
        }
        catch (EOFException e) {
               System.out.println(">>>> EOF is reached");
        }
    }




No comments:

Post a Comment

Total Pageviews

Popular Posts