TextureView autorotate video stream on Nexus4 (but not on Galaxy S4)

Question

When playback a video stream on TextureView, I saw different behaviors in Galaxy S4 and Nexus4.

The video source is recorded by Android in portrait mode, thus video is 90 degrees rotated. (Android camera's default orientation is landscape so to capture the video in portrait, we need to rotate video by 90 degrees clockwise) So when playback I first check the orientation of the video (by measuring width and height) and if width is longer than height, rotate TextureView by 90 degrees by below code.

MediaMetadataRetriever mediaMetadataRetriever = new MediaMetadataRetriever();
mediaMetadataRetriever.setDataSource(url, new HashMap<String, String>());
String heightString = mediaMetadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT);
String widthString = mediaMetadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH);

int videoHeight = Integer.parseInt(heightString);
int videoWidth = Integer.parseInt(widthString);

FrameLayout.LayoutParams l;
DisplayMetrics metrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(metrics);

if(videoWidth > videoHeight){
    Log.d(TAG, "Need to rotate by 90");

    l = new FrameLayout.LayoutParams(metrics.heightPixels, metrics.widthPixels, Gravity.CENTER_HORIZONTAL | Gravity.CENTER_VERTICAL);
    textureView.setRotation(90);            
}else{   
    l = new FrameLayout.LayoutParams(metrics.widthPixels, metrics.heightPixels, Gravity.CENTER_HORIZONTAL | Gravity.CENTER_VERTICAL);
}       

textureView.setLayoutParams(l);

This works on Galaxy S4 or Xepria Z, but doesn't work on Nexus4 or Galaxy S2. Seems like in Nexus4, TextureView autorotates video by 90 degrees, therefore together with below code, the video is in the end rotated by 180 degrees.

Right now I have no way to manage this behavior difference among devices. Does anyone have same experience and know how to solve it, I really appreciate your help.