Question

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

score 4.3k · Answer 1 · 2025-01-04 08:23:53

4.3k

Views

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

Published on 04 January 2025 at 08:23

score 271 · Answer 2 · 2025-01-04 08:25:50

271

Views

S3FS fails when given aiobotocore session to initialize the File System

Published on 04 January 2025 at 08:25

score 81 · Answer 3 · 2025-01-04 08:22:07

81

Views

Making s3fs work alongside ProcessPoolExecutor

Published on 04 January 2025 at 08:22

score 136 · Answer 4 · 2025-01-04 08:22:06

136

Views

How to use S3FS copy with recursive on folders?

Published on 04 January 2025 at 08:22

score 1.4k · Answer 5 · 2025-01-04 08:25:04

1.4k

Views

Can you use xr.open_mfdataset when reading files from S3 via s3fs?

Published on 04 January 2025 at 08:25

score 3k · Answer 6 · 2025-01-04 08:27:18

3k

Views

How to stream a large gzipped .tsv file from s3, process it, and write back to a new file on s3?

Published on 04 January 2025 at 08:27

score 815 · Answer 7 · 2025-01-04 08:25:55

815

Views

programmatically deleting parquet partitions from S3 bucket using pyspark

Published on 04 January 2025 at 08:25

score 1.3k · Answer 8 · 2025-01-04 08:27:08

1.3k

Views

Why do I get ConnectionResetError when reading and writing from and to s3 using smart_open?

Published on 04 January 2025 at 08:27

score 3.8k · Answer 9 · 2025-01-04 08:19:35

3.8k

Views

s3fs timeout on big S3 files

Published on 04 January 2025 at 08:19

score 1.7k · Answer 10 · 2025-01-04 08:26:35

1.7k

Views

Pandas 1.4.2 gives errors for installing s3fs while reading csv from S3 bucket

Published on 04 January 2025 at 08:26

score 690 · Answer 11 · 2025-01-04 08:20:36

690

Views

s3fs local filecache of versioned flies

Published on 04 January 2025 at 08:20

score 1.1k · Answer 12 · 2025-01-04 08:25:04

1.1k

Views

Read timeout in pd.read_parquet from S3, and understanding configs

Published on 04 January 2025 at 08:25

score 5.2k · Answer 13 · 2025-01-07 07:05:55

5.2k

Views

cannot import s3fs in pyspark

Published on 07 January 2025 at 07:05

score 1.7k · Answer 14 · 2025-01-04 08:20:08

1.7k

Views

use AWS_PROFILE in pandas.read_parquet

Published on 04 January 2025 at 08:20

score 596 · Answer 15 · 2025-01-04 08:23:16

596

Views

h5py slow when reading through an s3fs file object

Published on 04 January 2025 at 08:23

score 432 · Answer 16 · 2025-01-04 08:26:19

432

Views

Do I need to check integrity use pandas to upload and download file from s3?

Published on 04 January 2025 at 08:26

score 287 · Answer 17 · 2025-01-04 08:23:16

287

Views

Accessing S3 bucket object in TFX pipeline with S3FS

Published on 04 January 2025 at 08:23

score 222 · Answer 18 · 2025-01-04 08:24:08

222

Views

Want to create S3 Filesystem in python with stable library

Published on 04 January 2025 at 08:24

score 1.1k · Answer 19 · 2025-01-04 08:25:42

1.1k

Views

How to connect python s3fs client to a running Minio docker container?

Published on 04 January 2025 at 08:25

score 3.7k · Answer 20 · 2025-01-04 08:19:32

3.7k

Views

Attempting to Cache s3 files

Published on 04 January 2025 at 08:19

List Question

Cannot read parquet files in s3 bucket with Pyspark 2.4.4

S3FS fails when given aiobotocore session to initialize the File System

Making s3fs work alongside ProcessPoolExecutor

How to use S3FS copy with recursive on folders?

Can you use xr.open_mfdataset when reading files from S3 via s3fs?

How to stream a large gzipped .tsv file from s3, process it, and write back to a new file on s3?

programmatically deleting parquet partitions from S3 bucket using pyspark

Why do I get ConnectionResetError when reading and writing from and to s3 using smart_open?

s3fs timeout on big S3 files

Pandas 1.4.2 gives errors for installing s3fs while reading csv from S3 bucket

s3fs local filecache of versioned flies

Read timeout in pd.read_parquet from S3, and understanding configs

cannot import s3fs in pyspark

use AWS_PROFILE in pandas.read_parquet

h5py slow when reading through an s3fs file object

Do I need to check integrity use pandas to upload and download file from s3?

Accessing S3 bucket object in TFX pipeline with S3FS

Want to create S3 Filesystem in python with stable library

How to connect python s3fs client to a running Minio docker container?

Attempting to Cache s3 files

Trending Questions

Popular # Hahtags

Popular Questions