Working on a personal project to use the media projection manager API to create an Android app that allows me to take a screenshot whenever I say, 'screenshot' and then convert it to bitmap and sends it off to another function for processing. Which then recognizes what's in the image and speaks it aloud. However, the documentation is not good, and the examples have looked at on forums have led me to this code which still doesn't work. Firstly it throws several errors in Android studio. But even if I try to solve those at best I just get a black bitmap or the app crashes. I've got a foreground service running in the background that calls the screen capture service. Here's what the screencaptureservice looks like currently. Been working on it for a week and it's a real headache.
class ScreenshotService : Service() {
private lateinit var mediaProjectionManager: MediaProjectionManager
private lateinit var mediaProjection: MediaProjection
private lateinit var imageReader: ImageReader
private lateinit var handler: Handler
override fun onBind(intent: Intent?): IBinder? {
return null
}
override fun onCreate() {
super.onCreate()
// Initialize MediaProjectionManager and create a Handler on the main looper
mediaProjectionManager = getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager
handler = Handler(Looper.getMainLooper())
}
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
// Start the media projection when the service is started
startMediaProjection()
return START_STICKY
}
private fun startMediaProjection() {
// Retrieve the resultCode and data Intent extras passed to the service
val resultCode = intent?.getIntExtra("resultCode", -1)
val data = intent?.getParcelableExtra<Intent>("data")
if (resultCode != -1 && data != null) {
// Get a MediaProjection instance using the MediaProjectionManager
mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, data)
val windowManager = getSystemService(Context.WINDOW_SERVICE) as WindowManager
val metrics = DisplayMetrics()
windowManager.defaultDisplay.getMetrics(metrics)
// Set up an ImageReader to capture screenshots
imageReader = ImageReader.newInstance(metrics.widthPixels, metrics.heightPixels, PixelFormat.RGBA_8888, 2)
mediaProjection.createVirtualDisplay(
"Screenshot",
metrics.widthPixels,
metrics.heightPixels,
metrics.densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
imageReader.surface,
null,
null
)
// Start capturing screenshots in the background
captureScreenshots()
}
}
private fun captureScreenshots() {
imageReader.setOnImageAvailableListener({ reader ->
val image = reader.acquireLatestImage()
if (image != null) {
// Process the screenshot here
val bitmap = imageToBitmap(image)
saveBitmapToFile(bitmap)
image.close()
}
}, handler)
}
private fun imageToBitmap(image: Image): Bitmap {
val planes = image.planes
val buffer: ByteBuffer = planes[0].buffer
val pixelStride: Int = planes[0].pixelStride
val rowStride: Int = planes[0].rowStride
val rowPadding = rowStride - pixelStride * image.width
val bitmap = Bitmap.createBitmap(
image.width + rowPadding / pixelStride,
image.height,
Bitmap.Config.ARGB_8888
)
// Copy the image data to the bitmap
bitmap.copyPixelsFromBuffer(buffer)
return bitmap
}
private fun saveBitmapToFile(bitmap: Bitmap) {
// Save the bitmap to a file or perform any desired action
try {
val fos = FileOutputStream("/path/to/screenshot.png")
bitmap.compress(Bitmap.CompressFormat.PNG, 100, fos)
fos.close()
} catch (e: IOException) {
e.printStackTrace()
}
}
}
Any help appreciated, I know it's easier to just use a screen reader, etc, but that's not what I'm trying to do. Love challenges but still pretty new to android