gzip

How to get gzip operating recursively at all depths?

How to get gzip operating recursively at all depths? I mean to gzip all files *.vtu, at all depths below a given directory, in bash. I have such files at depths 1 and 2 below ./. I managed to do so with $ gzip -v $(find . -name "*.vtu") I could also use find … …

Total answers: 3

How to modify a gzipped file with sed and then zip again the file?

How to modify a gzipped file with sed and then zip again the file? I have a .vcf.gz file, with the following aspect: #CHROM POS ID REF ALT chr1 10894 chr1:10894:G:A G A chr1 10915 chr1:10915:G:A G A chr1 10930 chr1:10930:G:A G A I want to modify the CHROM column to remove ‘chr’ and to …

Total answers: 2

Rename .gz files according to names in separate txt-file

Rename .gz files according to names in separate txt-file I want to rename .gz files according to names in separate txt-file. I have a map with .gz files with the names: trooper10.gz trooper11.gz trooper12.gz etc. and I have a separate txt-file with the wanted name(s) in in the first column and the .gz-names in the …

Total answers: 2

Normal gz file not extractable by tar

Normal gz file not extractable by tar I have a gz archive but for some reason tar said that the format is incorrect even though I can double click it in mac Finder and extract it normally, and file command shows the same format just like any other tar.gz files Why is that and how …

Total answers: 1

Extract first n bytes from .tar.gz and output as a .tar.gz in a single command

Extract first n bytes from .tar.gz and output as a .tar.gz in a single command I have a .tar.gz as input and want to extract the first 128 MiB of it and output as a .tar.gz in a single command. I tried: sudo tar xzOf input.tar.gz | sudo dd of=output bs=1M count=128 iflag=fullblock | sudo …

Total answers: 1

How to backup many large files to single compressed file on S3

How to backup many large files to single compressed file on S3 I have an application that has many thousands of files totaling over 10TB. I’d need to backup this data somewhere (probably to AWS S3). I’d like to: compress data being backed up save the backup as a single file For example as a …

Total answers: 1

Why tar command uses gzip command through 'z' option?

Why tar command uses gzip command through 'z' option? In Linux Ubuntu about the ‘tar’ command for these versions: tar -tzf /path/to/filename.tar.gz # Show the content tar -xzf /path/to/filename.tar.gz # Extract the content Observe both commands use the z option, and well, they work as expected. Through man tar, about the z option, it indicates: …

Total answers: 1

Unzip compressed dump and import via psql

Unzip compressed dump and import via psql I’m trying to dump a huge database and compress the dump in order to not have to wait hours till it’s done. I dump the database the following way: pg_dump -Fc -U -v | gzip > db$(date +%d-%m-%y_%H-%M).tar.gz This leaves me with a compressed tar file. I know …

Total answers: 1

gunzip multiple gz files with same compressed file name in multiple folders

gunzip multiple gz files with same compressed file name in multiple folders I have a directory where there are multiple folders, each folder contains multiple .gz files with the same zipped file name "spark.log". How can I unzip all of them at once and rename them like the gz file? My data looks like this …

Total answers: 2

Convert dos line endings in .gz zipped file faster

Convert dos line endings in .gz zipped file faster I have 120 .gz files (each about 5G) with dos line endings, my goal is to convert those to unix line endings, but I don’t want to wait multiple days. Here is my current approach: function conv() { tmpfile=$(mktemp .XXXXXX) zcat $1 > $tmpfile dos2unix $tmpfile …

Total answers: 1

Why GZIP utility cares about extension?

Why GZIP utility cares about extension? Whenever I need to decompress a file which was compressed with Gzip, I need to rename the file with a .gz extension , I tried other compression utility like zip, bzip2 and they don’t seem to care about extensions. Why is this is the case with Gzip , isn’t …

Total answers: 1

Pipe find list of files into xargs gzip and pipe again into pigz

Pipe find list of files into xargs gzip and pipe again into pigz I need to find files newer than x days, and then turn it into a gzip, but I want to do it using pigz. For now I’m now doing it the slow way; this works: find /path/to/src -type f -mtime -90 | …

Total answers: 2

Tarring-gzipping only hidden directories (or files + directories)

Tarring-gzipping only hidden directories (or files + directories) Based on this, to tar all hidden files in the current directory one can use ls -A | egrep ‘^.’ | tar cvf ./test.tar -T – However, how can one tar only all hidden directories or all hidden directories and files in the current directory? Based on …

Total answers: 2

How to uncompress a gzipped partition image and dd it directly to the destination partition without writing to current partition?

How to uncompress a gzipped partition image and dd it directly to the destination partition without writing to current partition? I’ve backed up a partition using sudo dd bs=8M if=/dev/sda2 | gzip > /someFolderOnSDB/sda2.img.gz. The image is stored on a separate disk sdb. When restoring it using gunzip -k /mnt/bkp/sda2.img.gz | sudo dd of=/dev/sda2, I …

Total answers: 1

unzip .tar.gz file in terminal

unzip .tar.gz file in terminal How do I unzip a .tar.gz file in the terminal? I’m not able to do so in GUI, because I want to extract it to a /usr directory, and will need the sudo command. Asked By: user801855 || Source gzip -d this_is_the_file.tar.gz tar -xvf this_is_the_file.tar These can be combined, but …

Total answers: 1

How portable is a gzip file over 4 GB in size?

How portable is a gzip file over 4 GB in size? To backup a snapshot of my work, I run a command like tar -czf work.tgz work to create a gzipped tar file, which I can then drop in cloud storage. However, I have just noticed that gzip has a 4 GB size limit, and …

Total answers: 2

GZip doesn't produce the same compressed result on macOS vs Linux

GZip doesn't produce the same compressed result on macOS vs Linux I have a few thousand files that are individually GZip compressed (passing of course the -n flag so the output is deterministic). They then go into a Git repository. I just discovered that for 3 of these files, Gzip doesn’t produce the same output …

Total answers: 4