Combining ANSI Color codes — Foreground and background color merging

I have this function in zsh to run grep on the output of git log to highlight the line with the commit’s hash, author, message, etc.. I’m rewriting to try and add color to the output and want to prevent grep from overwriting the foreground color that’s passed.
I’ve tried all variations of --color=(always|never|auto). This is the current command I’m stuck with:

# raw command:
unbuffer git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' | grep -i '.*'update'.*' | column -t -s $'t'

# w/ comments:
unbuffer # From `expect` apt package, preserves color across piping by pushing to a buffer? idk.

# git log, formatted
git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)'

grep -i '.*'update'.*' # grep `update` from any line in input, hardcoded here but could also be a parameter

column -t -s $'t' # tabulate

The output looks like this.

The output looks like this

In addition, piping to column to tabulate the output without the use of unbuffer also washes away the color and there’s no --color=never option for the column command. The use of unbuffer also causes grep‘s coloring to not work at all.

What I want is to just have the background of the grep match to be highlighted red, green, or whatever color while preserving the color of the text as seen in the image, i.e. green on red, blue on red, etc. I’m not completely familiar with how terminals work under the hood and why color is lost when piping.

I’m sure I could write a C++ program to do all this by inserting ANSI color codes where necessary but I figured I would at least ask. I do want to only use commands that can be installed via apt as this is a part of my whole terminal config that I sync across multiple devices via a git repo..

Edits below this point:

  • I originally tried to include a -A1 -B1 flag with grep to show the commit before and after, hence the desire for explicit highlighting.
  • The answer from @daniel-t works for resolving coloring being lost by git log, but disables grep from highlighting matches.

I am able to achieve the desired effect by writing color codes manually into a string and echoing via echo "e[1;34m13ca348e[m e[1;36mSun, 29 Jane[m -- e[1;41me[1;32mCommit messagee[m". This output is exactly what I’m looking for (notice the green on red from wrapping the foreground and background color codes) but I doubt terminal programs structure colors this way so I don’t know if another program would be able combine them easily this way .

Asked By: Sargates


Last solution

Try git log --color --pretty="$(printf 'e[1;34m%%h e[36m%%aDe[35Ge[;2;37mt%%ane[70Ge[;37m%%se[m')" | GREP_COLORS='ms=1;32;41:sl=37' grep -Pi 'e[;37m.*K''update' --color=always. I eliminated the use of column and sed. The new e[70G just moves the cursor to column 70, and that should be enough for names. The first two columns are more or less fixed-length and don’t need much.

last solution

Simple solution

Try git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(bold green red)%s%C(reset)' --color=always | grep -i '.*'update'.*' | column -t -s $'t'. I just changed the last "white" to "bold green red".

highlight entire commit message

Based on your desired color code example highlighting the entire commit message, it was easy to color the entire last column. There is no need for combining colors from two commands. I don’t know which version of grep you are using that has red as the background, but the grep on Ubuntu has red as the foreground, so we needed to override this.

Advanced solution

EDIT: Based on what you seem to want:

The command becomes git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' --color=always | GREP_COLORS='ms=1;41;32:sl=37' grep -Pi 'e[37m.*K''update' --color=always | column -t -s $'t'. Edit the 'update' part to choose which pattern to search for.

highlight only matched part

The GREP_COLORS environment variable controls how grep highlights and is separated by :. ms=1;32;41 applies bold green on red, respectively. grep will reset the colors immediately after, so we need to add sl=37 to get back the white (necessary on xterm but not most terminals like Konsole). ANSI color escape sequences would normally merge with the previous foreground, background, or style if omitted, but you can’t push/pop/undo so that doesn’t work for your case. I then match for e[37m (requires -P) to restrict us to the commit message (to avoid glitchy colors in case it matches elsewhere), relying on the fact that we only use white in git once. Then the .*K tells grep to search for your keyword after, and to avoid highlighting and ruining the previous color code itself. Finally we need --color=always to enable our new colors.

By the way, you don’t need to use reset so much. Here is a shorter version git log --color --pretty=$'%C(bold blue)%h t%C(cyan)%aDt%C(reset dim white)t%ant%C(reset white)%s%C(reset)' | GREP_COLORS='ms=1;32;41:sl=37' grep -Pi 'e[;37m.*K''update' --color=always | column -t -s $'t'. The middle ones are still there because they are more reliable than no-dim (same color code outputted as no-bold), and the last one is only needed if you don’t have a colored bash prompt.

Old answer

You need to pass --color=always to git, not grep or column. Try git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' --color=always | grep -i '.*'update'.*' | column -t -s $'t'.

I get the following output:

home@daniel-tablet1:~/WebstormProjects/linuxenv$ git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' --color=always | grep -i '.*'update'.*' | column -t -s $'t'
13ca348   Sun, 29 Jan 2023 15:56:02 -0500    Daniel Tang  Update PlantUML 1.2022.6 -> 1.2023.0
83b31a2   Sun, 29 Jan 2023 15:48:21 -0500    Daniel Tang  Update Gradle 7.5.1 -> 7.6
85f2b04   Sun, 29 Jan 2023 15:45:28 -0500    Daniel Tang  Update GraalVM 22.3.0 -> 22.3.1
5e86df9   Sun, 29 Jan 2023 15:39:59 -0500    Daniel Tang  Update JetBrainsMono 2.242 -> 2.304
7e984d4   Wed, 26 Oct 2022 18:21:23 -0400    Daniel Tang  Update graalvm 22.3
ee202a6   Wed, 12 Oct 2022 10:00:33 -0400    Daniel Tang  Update rustup settings.toml hash
6a0c58f   Mon, 19 Sep 2022 23:54:54 -0400    Daniel Tang  Update
a9755a3   Tue, 23 Mar 2021 13:54:27 -0400    Daniel Tang  Fix nvmupdate
e936dab   Fri, 19 Mar 2021 17:29:32 -0400    Daniel Tang  Update
0b8e7e3   Wed, 17 Mar 2021 04:19:11 -0400    Daniel Tang  Update
(sky blue)(turquoise                    )    (dark grey)  (white) (konsole)
(RGB blue)(cyan                         )    (dark grey)  (light gray) (xterm)
(navyblue)(aquamarine                   )    (dark grey)  (light gray) (gnome-terminal)

I removed unbuffer because it not only needed to be installed, but also causes the command to hang on my computer. The standard command for what you desire is stdbuf -o0. If you say that unbuffer was fixing the colors, that must be because it was pretending to be a terminal by allocating a PTY. That is unnecessary if we use the more efficient direct solution of --color=always.

If you wanted to find out which command was failing to pass on the color, you can use cat. git log --pretty=$'%C(bold blue)%h%C(reset) t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' | cat results in missing color, and without the cat there is paging and color. This shows that it’s git that fails to emit the ANSI color codes. Even if you wrote a C++ program, those are your own color code, and no color codes from git.

Don’t worry about grep overwriting your color code. Your .*'update'.* goes to the start of the line. A color code is specially chosen text with escape sequences. Even if grep appended any color codes, it will be overwritten by git‘s color code immediately after. The head -n1 output of the git --color=always is e[1;34m9a8c818e[m te[1;36mMon. In this case, it’s good that grep did not decided to insert color codes because it was pipe to something, namely column after, but even if you gave grep --color=always, it just prepends it color codes. I get e[01;31me[Ke[1;34m13ca348e[m te[1;36mSun, 29 Jan. It appears the same visually because the grep‘s colors are immediately overwritten by git‘s colors from earlier.

Answered By: Daniel T

So I was able to find a solution. Setting GREP_COLORS inline with the grep call allowed me to change grep to only highlight the background without conflicting with the foreground coloring from git log. I also had to pass --color=always in grep to prevent washing when piped to column like so: GREP_COLORS="ms=41" grep -i update --color=always

See: for a table on GREP_COLORS placeholders

Also the issue with column is that it doesn’t support every ANSI color escape and ends up washing any it doesn’t like. My fix for this was to use column_ansi and that acts as a drop-in replacement (mostly).

Like I said in the edit, I was trying to include the flags -A1 -B1 in grep to get the commit before and after.

Here is the full command I ended up using and I ended up wrapping it in a function:

glgrep() {
    git log --color=always --pretty=format:$'%C(bold blue)%h%C(reset)t%C(bold cyan)%aD%C(reset)t%C(dim white)t%an%C(reset)t%C(white)%s%C(reset)' | GREP_COLORS="ms=41" grep -i $1 --color=always -A1 -B1 | column_ansi -t -s $'t' | sed 's/^[ t]*//;s/[ t]*$//'
    # call after `column_ansi` requires trimming. i dont know why (extra following newline char?)
Answered By: Sargates
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.