WebPack 4 fails to load `tsx` file for React: `You may need an appropriate loader to handle this file type.`

Question

Summary:

After creating a React TypeScript application and configuring WebPack 4, I get the following error when it's trying to parse a tsx file:

ERROR in ./src/App.tsx 6:16
Module parse failed: Unexpected token (6:16)
You may need an appropriate loader to handle this file type.

The verbose output doesn't indicate which loader it's attempting to use, nor which file pattern that matched.

How can I troubleshoot this problem?

I opened a bug for shitty error messages here: https://github.com/webpack/webpack/issues/9025

Here's how to reproduce:

I created a react application like this:

npx create-react-app kontakt-manager --typescript

Then I added this WebPack 4 configuration file, and installed the required packages:

const path = require('path');

module.exports = {
    // change to .tsx if necessary
    entry: './src/App.tsx',
    output: {
        filename: 'bundle.js',
        path: path.resolve(__dirname, 'dist')
    },
    resolve: {
        // changed from extensions: [".js", ".jsx"]
        extensions: ['.ts', '.tsx', '.js', '.jsx']
    },
    module: {
        rules: [
            // changed from { test: /\.jsx?$/, use: { loader: 'babel-loader' } },
            { test: /\.(t|j)sx?$/, use: { loader: 'awesome-typescript-loader' } },
            // addition - add source-map support
            { enforce: "pre", test: /\.js$/, loader: "source-map-loader" }
        ]
    },
    externals: {
        "react": "React",
        "react-dom": "ReactDOM",
    },
    // addition - add source-map support
    devtool: "source-map"
}

Then I try to package everything:

$ npx webpack --verbose
ℹ ｢atl｣: Using [email protected] from typescript
ℹ ｢atl｣: Using tsconfig.json from <directory>/tsconfig.json
ℹ ｢atl｣: Checking started in a separate process...
ℹ ｢atl｣: Time: 67ms
Hash: 47205d90a8bf0ab03285
Version: webpack 4.29.6
Time: 2404ms
Built at: 2019-04-10 12:13:19 a.m.
 2 assets
Entrypoint main = bundle.js bundle.js.map
chunk    {0} bundle.js, bundle.js.map (main) 332 bytes [entry] [rendered]
    > ./src/App.tsx main
 [0] ./src/App.tsx 332 bytes {0} [depth 0] [built] [failed] [1 error]
     ModuleConcatenation bailout: Module is not an ECMAScript module
     single entry ./src/App.tsx  main

WARNING in configuration
The 'mode' option has not been set, webpack will fallback to 'production' for this value. Set 'mode' option to 'development' or 'production' to enable defaults for each environment.
You can also set it to 'none' to disable any default behavior. Learn more: https://webpack.js.org/concepts/mode/

ERROR in ./src/App.tsx 6:16
Module parse failed: Unexpected token (6:16)
You may need an appropriate loader to handle this file type.
| class App extends Component {
|     render() {
>         return (<div className="App">
|         <header className="App-header">
|           <img src={logo} className="App-logo" alt="logo"/>