How to modify a gzipped file with sed and then zip again the file? I have a .vcf.gz file, with the following aspect: #CHROM POS ID REF ALT chr1 10894 chr1:10894:G:A G A chr1 10915 chr1:10915:G:A G A chr1 10930 chr1:10930:G:A G A I want to modify the CHROM column to remove ‘chr’ and to …
Rename .gz files according to names in separate txt-file I want to rename .gz files according to names in separate txt-file. I have a map with .gz files with the names: trooper10.gz trooper11.gz trooper12.gz etc. and I have a separate txt-file with the wanted name(s) in in the first column and the .gz-names in the …
Normal gz file not extractable by tar I have a gz archive but for some reason tar said that the format is incorrect even though I can double click it in mac Finder and extract it normally, and file command shows the same format just like any other tar.gz files Why is that and how …
Extract first n bytes from .tar.gz and output as a .tar.gz in a single command I have a .tar.gz as input and want to extract the first 128 MiB of it and output as a .tar.gz in a single command. I tried: sudo tar xzOf input.tar.gz | sudo dd of=output bs=1M count=128 iflag=fullblock | sudo …
Difference between ar, tar, gzip, zip and when should I decide to choose which one? I had a use case where I needed to pack a bunch of files into one. And all above commands does the same. I know gzip compresses my files but lets say space is not at all an issue for …
How to backup many large files to single compressed file on S3 I have an application that has many thousands of files totaling over 10TB. I’d need to backup this data somewhere (probably to AWS S3). I’d like to: compress data being backed up save the backup as a single file For example as a …
Why tar command uses gzip command through 'z' option? In Linux Ubuntu about the ‘tar’ command for these versions: tar -tzf /path/to/filename.tar.gz # Show the content tar -xzf /path/to/filename.tar.gz # Extract the content Observe both commands use the z option, and well, they work as expected. Through man tar, about the z option, it indicates: …
Unzip compressed dump and import via psql I’m trying to dump a huge database and compress the dump in order to not have to wait hours till it’s done. I dump the database the following way: pg_dump -Fc -U -v | gzip > db$(date +%d-%m-%y_%H-%M).tar.gz This leaves me with a compressed tar file. I know …
gunzip multiple gz files with same compressed file name in multiple folders I have a directory where there are multiple folders, each folder contains multiple .gz files with the same zipped file name "spark.log". How can I unzip all of them at once and rename them like the gz file? My data looks like this …
Convert dos line endings in .gz zipped file faster I have 120 .gz files (each about 5G) with dos line endings, my goal is to convert those to unix line endings, but I don’t want to wait multiple days. Here is my current approach: function conv() { tmpfile=$(mktemp .XXXXXX) zcat $1 > $tmpfile dos2unix $tmpfile …
How do I .zip multiple files and retain their file names. EX file1.file file2.file -> file1.zip file2.zip As the title suggests I need to .zip multiple files (thousands) in a folder and have each file zipped into their own archive with the original file name as the .zip filename. If I try to do this …
Why GZIP utility cares about extension? Whenever I need to decompress a file which was compressed with Gzip, I need to rename the file with a .gz extension , I tried other compression utility like zip, bzip2 and they don’t seem to care about extensions. Why is this is the case with Gzip , isn’t …
Pipe find list of files into xargs gzip and pipe again into pigz I need to find files newer than x days, and then turn it into a gzip, but I want to do it using pigz. For now I’m now doing it the slow way; this works: find /path/to/src -type f -mtime -90 | …
Tarring-gzipping only hidden directories (or files + directories) Based on this, to tar all hidden files in the current directory one can use ls -A | egrep ‘^.’ | tar cvf ./test.tar -T – However, how can one tar only all hidden directories or all hidden directories and files in the current directory? Based on …
How to uncompress a gzipped partition image and dd it directly to the destination partition without writing to current partition? I’ve backed up a partition using sudo dd bs=8M if=/dev/sda2 | gzip > /someFolderOnSDB/sda2.img.gz. The image is stored on a separate disk sdb. When restoring it using gunzip -k /mnt/bkp/sda2.img.gz | sudo dd of=/dev/sda2, I …
unzip .tar.gz file in terminal How do I unzip a .tar.gz file in the terminal? I’m not able to do so in GUI, because I want to extract it to a /usr directory, and will need the sudo command. Asked By: user801855 || Source gzip -d this_is_the_file.tar.gz tar -xvf this_is_the_file.tar These can be combined, but …
How portable is a gzip file over 4 GB in size? To backup a snapshot of my work, I run a command like tar -czf work.tgz work to create a gzipped tar file, which I can then drop in cloud storage. However, I have just noticed that gzip has a 4 GB size limit, and …
GZip doesn't produce the same compressed result on macOS vs Linux I have a few thousand files that are individually GZip compressed (passing of course the -n flag so the output is deterministic). They then go into a Git repository. I just discovered that for 3 of these files, Gzip doesn’t produce the same output …
Is there a compression tool with an arbitrarily large dictionary? I am looking for a compression tool with an arbitrarily large dictionary (and “block size”). Let me explain by way of examples. First let us create 32MB random data and then concatenate it to itself to make a file of twice the length of length …
Uncompressed file estimation wrong? I had a large (~60G) compressed file (tar.gz). I used split to break it into 4 parts and then cat to join them back together. However, now, when I am trying to estimate the size of the uncompressed file, it turns out it is smaller than the original? How is this …
