Minimal reproducible code:
@override
Widget build(BuildContext context) {
return Padding(
padding: const EdgeInsets.all(40),
child: Scaffold(),
);
}
I see black background behind my Scaffold, obviously this isn't Theme.of(context).scaffoldBackgroundColor because both light and dark theme show same white color. So, what color is it?

There is no property to define that color inside ThemeData. You can check at: https://api.flutter.dev/flutter/material/ThemeData-class.html
As you say, the property scaffoldBackgroundColor is just the color of the Material widget that underlies the entire Scaffold.
But if you want a color or maybe a gradient behind your Scaffold, you can use container.
Solid Color
Linear Gradient