Question

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

score 4.3k · Answer 1 · 2025-01-04 08:23:53

4.3k

Views

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

Published on 04 January 2025 at 08:23

score 260 · Answer 2 · 2025-01-04 08:25:50

260

Views

S3FS fails when given aiobotocore session to initialize the File System

Published on 04 January 2025 at 08:25

score 73 · Answer 3 · 2025-01-04 08:22:07

73

Views

Making s3fs work alongside ProcessPoolExecutor

Published on 04 January 2025 at 08:22

score 128 · Answer 4 · 2025-01-04 08:22:06

128

Views

How to use S3FS copy with recursive on folders?

Published on 04 January 2025 at 08:22

score 1.4k · Answer 5 · 2025-01-04 08:25:04

1.4k

Views

Can you use xr.open_mfdataset when reading files from S3 via s3fs?

Published on 04 January 2025 at 08:25

score 3k · Answer 6 · 2025-01-04 08:27:18

3k

Views

How to stream a large gzipped .tsv file from s3, process it, and write back to a new file on s3?

Published on 04 January 2025 at 08:27

score 806 · Answer 7 · 2025-01-04 08:25:55

806

Views

programmatically deleting parquet partitions from S3 bucket using pyspark

Published on 04 January 2025 at 08:25

score 1.3k · Answer 8 · 2025-01-04 08:27:08

1.3k

Views

Why do I get ConnectionResetError when reading and writing from and to s3 using smart_open?

Published on 04 January 2025 at 08:27

score 3.8k · Answer 9 · 2025-01-04 08:19:35

3.8k

Views

s3fs timeout on big S3 files

Published on 04 January 2025 at 08:19

score 1.7k · Answer 10 · 2025-01-04 08:26:35

1.7k

Views

Pandas 1.4.2 gives errors for installing s3fs while reading csv from S3 bucket

Published on 04 January 2025 at 08:26

score 685 · Answer 11 · 2025-01-04 08:20:36

685

Views

s3fs local filecache of versioned flies

Published on 04 January 2025 at 08:20

score 1.1k · Answer 12 · 2025-01-04 08:25:04

1.1k

Views

Read timeout in pd.read_parquet from S3, and understanding configs

Published on 04 January 2025 at 08:25

score 5.2k · Answer 13 · 2025-01-07 07:05:55

5.2k

Views

cannot import s3fs in pyspark

Published on 07 January 2025 at 07:05

score 1.7k · Answer 14 · 2025-01-04 08:20:08

1.7k

Views

use AWS_PROFILE in pandas.read_parquet

Published on 04 January 2025 at 08:20

score 592 · Answer 15 · 2025-01-04 08:23:16

592

Views

h5py slow when reading through an s3fs file object

Published on 04 January 2025 at 08:23

score 427 · Answer 16 · 2025-01-04 08:26:19

427

Views

Do I need to check integrity use pandas to upload and download file from s3?

Published on 04 January 2025 at 08:26

score 282 · Answer 17 · 2025-01-04 08:23:16

282

Views

Accessing S3 bucket object in TFX pipeline with S3FS

Published on 04 January 2025 at 08:23

score 217 · Answer 18 · 2025-01-04 08:24:08

217

Views

Want to create S3 Filesystem in python with stable library

Published on 04 January 2025 at 08:24

score 1.1k · Answer 19 · 2025-01-04 08:25:42

1.1k

Views

How to connect python s3fs client to a running Minio docker container?

Published on 04 January 2025 at 08:25

score 3.7k · Answer 20 · 2025-01-04 08:19:32

3.7k

Views

Attempting to Cache s3 files

Published on 04 January 2025 at 08:19

List Question

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

S3FS fails when given aiobotocore session to initialize the File System

Making s3fs work alongside ProcessPoolExecutor

How to use S3FS copy with recursive on folders?

Can you use xr.open_mfdataset when reading files from S3 via s3fs?

How to stream a large gzipped .tsv file from s3, process it, and write back to a new file on s3?

programmatically deleting parquet partitions from S3 bucket using pyspark

Why do I get ConnectionResetError when reading and writing from and to s3 using smart_open?

s3fs timeout on big S3 files

Pandas 1.4.2 gives errors for installing s3fs while reading csv from S3 bucket

s3fs local filecache of versioned flies

Read timeout in pd.read_parquet from S3, and understanding configs

cannot import s3fs in pyspark

use AWS_PROFILE in pandas.read_parquet

h5py slow when reading through an s3fs file object

Do I need to check integrity use pandas to upload and download file from s3?

Accessing S3 bucket object in TFX pipeline with S3FS

Want to create S3 Filesystem in python with stable library

How to connect python s3fs client to a running Minio docker container?

Attempting to Cache s3 files

Trending Questions

Popular # Hahtags

Popular Questions