How to see the first few lines from a file in S3 using AWS CLI? - Big Data In Real World

How to see the first few lines from a file in S3 using AWS CLI?

How to solve word count problem in Hive?
April 9, 2021
How to specify conditional expressions (OR, AND, NOT) when searching documents in Elasticsearch?
April 14, 2021
How to solve word count problem in Hive?
April 9, 2021
How to specify conditional expressions (OR, AND, NOT) when searching documents in Elasticsearch?
April 14, 2021

This is a very common requirement and it has an easy solution.

Solution

aws s3api get-object to get the object and use to –range option to fetch only the specified number of bytes.

Since we are just previewing the content, instead of writing to a file we can send the output to standard output by specifying /dev/stdout and followed by a pipe to head

[osboxes@wk1 ~]$ aws s3api get-object --bucket hirw-bucket-versions 
--key version-test --range bytes=0-1000 /dev/stdout | head

Hello, this is VERSION TWO

{
    "AcceptRanges": "bytes",
    "LastModified": "2020-11-18T19:40:35+00:00",
    "ContentLength": 27,
    "ETag": "\"04cf95a672923b5c021568765b336c4f\"",
    "VersionId": "8LKeiP26.7WS_CUYUmY_BNmpIX0ljIxa",
    "ContentRange": "bytes 0-26/27",
    "ContentType": "binary/octet-stream",
    "Metadata": {}

 

Big Data In Real World
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.

Comments are closed.

How to see the first few lines from a file in S3 using AWS CLI?
This website uses cookies to improve your experience. By using this website you agree to our Data Protection Policy.

Hadoop In Real World is now Big Data In Real World!

X