How to prevent "Warning: failure in signing Android apk" error when using adt to compile an Adobe AIR app

Question

I'm using the latest AIRSDK 50.2.2.5 from Harman to compile a captive APK and I get the message "Warning: failure in signing Android apk".

My .p12 file was created from Animate back in 2012, but it always worked with older versions of Adobe AIR. Curiosly enough, if I create a new .p12 from Animate, the compilation goes well without the "failure in signing" message, but the resulting APK is useless to be uploaded to the Play Store as an update because it says that the certificate used to sign the APK is different so I can't upload the app as an update.

So, how do I overcome this problem of "failure signing Android apk" while using my old .p12 certificate? I've opened it in the "Keystore Explorer" tool and it says it expires in 2036-05-17, so there's still plenty of time to use this certificate.

The only difference I see is that Keystore Explorer says the new .p12 certificate that works (but is useless for app updates) has a 2048 key size whereas the old .p12 has a 1024 key size. Since I noticed this difference, I tried to research a bit about this and found this article from Adobe:

https://helpx.adobe.com/mt/x-productkb/multi/rsa1024-certificate-issue-with-adt-tool.html

It says that "Packaging Android app fails with RSA-1024 certificates" on computers with Java 8 and higher. The only solution it suggests is downgrading to Java 7 or 6. I've tried to use Java 7 with adt (by configuring it in adt.cfg) but then I get the following messages:

"WARNING: could not find appropriate Java Home location for v11+ JDK

WARNING: could not find appropriate Java Home location for v8+ JDK"

So I can't use Java 7 or lower because ADT looks for newer versions, but I can't use Java 8 or higher because it failts with RSA-1024 .p12 certificates.

I've also tried adding the <BuildLegacyAPK>true</BuildLegacyAPK> flag inside the <android>...</android> block. But then I get another error message:

"SDK is missing file /Users/myuser/AndroidSDK/build-tools/33.0.2/lib/dx.jar"

After researching for this error message I found a possible solution would be creating copies of d8 and d8.jar and naming them "dx" and "dx.jar". But doing so yields YET another error message:

"dx tool failed:no main manifest attribute in /Users/myuser/AndroidSDK/build-tools/33.0.2/lib/dx.jar"

So after lots of wasted hours, I'm stumped. What should I do to be able to publish app updates with my old certificate again? There has to be some solution I couldn't find.

Answer 1

I found out how to fix this problem, so I'm posting here all the steps needed so other people can save a bit of time:

• Download and install the latest version of Android Studio (run it once to download the Android SDK) or install just the Android SDK with "sdkmanager"
• Download and install Java SE Development Kit 8u112 (find inside the page the exact text "Java SE Development Kit 8u112", since anything higher than that won't work with old certificates)
• Download and install Java SE Development Kit 11.0.x (needed as well to compile AAB packages with a new certificate; don't download a much higher version since too recent versions don't work with the AIR compiler)
• Download the specific version of the "pepk" tool that works with RSA-1024 certificates (newer versions won't work)
• Generate an encrypted version of the old certificate as well as a new RSA-2048 upload certificate as follows (commands tested in a macOS computer, but in other platforms it should be similar; if using Windows, set JAVA_HOME and the system PATH for either versions (8/11) of the Java command using these instructions instead of using "export").

# Use Java 8 Update 112 which works with old RSA-1024 certificates
export JAVA_HOME=`/usr/libexec/java_home -v1.8.0_112`
export JAVACMD=${JAVA_HOME}/bin/java

# Encrypt the old P12 certificate to upload to Google (the last argument is the 68 byte key Google provides in the "App integrity" > "App signing" section of its Play Console)
java -jar ~/path_to_pepk/pepk.jar --keystore=OLD_CERTIFICATE.p12 --alias=1 --output=OLD_CERTIFICATE.enc --encryptionkey=XXXXXXXX

# Create a new RSA-2048 certificate to be used as the upload certificate
adt -certificate -cn "COMMON_NAME" -ou "ORGANIZATIONAL_UNIT" -o "ORGANIZATION" -c "TWO_LETTER_COUNTRY_CODE" -validityPeriod 20 2048-RSA upload_certificate.p12 NEW_PASSWORD

# Convert the new P12 certificate to PEM
keytool -export -rfc -keystore upload_certificate.p12 -alias 1 -file upload_certificate.pem

• Upload both certificates to the Play Console by going to "App integrity" > "App signing" > "Export and upload a key from Java keystore" (or similarly titled option). Don't use the "Download PEPK tool" link in that page because that version won't work with RSA-1024 certificates. Use the version downloaded earlier instead. Upload the encrypted version of the old RSA-1024 key by clicking on "Upload private key" and also upload the new RSA-2048 upload certificate in the "Upload your upload key certificate" section below (although it says it's "optional", it's actually NOT optional in this case, since this is also needed in order to be able to use the RSA-2048 certificate for compilation instead of the old one).

• Finally add this to the compilation script before calling ADT so Java 11 is used for compilation of the AAB package (since Java 8 can't be used for that).

# Use Java 11 to create AAB packages with an RSA-2048 certificate
export JAVA_HOME=`/usr/libexec/java_home -v11`
export JAVACMD=${JAVA_HOME}/bin/java

• Create the AAB package using adt with -target aab argument and always using the new .p12 certificate for both AAB packages and test APKs (keep a backup of the old certificate just in case).