How to merge duplicated history of commands in interactive shell?

When you try to find a previous used command in an interactive shell session
by  (up arrow), you may get something like

$ls         # 1st time push `up arrow`
$ls         # 2nd time push `up arrow`
$ls         # 3rd time push `up arrow`
$ls         # 4th time push `up arrow`
$ls         # 5th time push `up arrow`
$ls         # 6th time push `up arrow`
$make       # 7th time push `up arrow`
$make       # 8th time push `up arrow`
$make       # 9th time push `up arrow`
$ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"   # Bingo!

I would like it better if it were like this:

$ls         # 1st time push `up arrow`
$make       # 2th time push `up arrow`
$ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"   # Bingo!

because the duplicated history is usually of no use.

How can I get Bash to do this?

Asked By: fronthem

||

You can accomplish this by setting ignoredups in the HISTCONTROL variable:

HISTCONTROL="ignoredups"

Optionally export it,

export HISTCONTROL="ignoredups"

to make it an environment variable. 
From the bash(1) man page:

HISTCONTROL

    A colon-separated list of values controlling how commands are
    saved on the history list. 
    If the list of values includes ignorespace,
    lines which begin with a space character are not saved in the history
    list.  A value of ignoredups causes lines matching the previous
    history entry to not be saved.  A value of ignoreboth is shorthand
    for ignorespace and ignoredups.  A value of erasedups causes all
    previous lines matching the current line to be removed from the
    history list before that line is saved.  Any value not in the above
    list is ignored.  If HISTCONTROL is unset, or does not include a valid
    value, all lines read by the shell parser are saved on the history
    list, subject to the value of HISTIGNORE
    The second and
    subsequent lines of a multi-line compound command are not tested,
    and are added to the history regardless of the value of
    HISTCONTROL.

This does exactly what the question asks for. 
If the user enters the commands ruby …,
make, make, make, ls, ls, ls, ls, ls and ls
(as separate, consecutive lines),
then the history list will be ruby …, make, ls
Pressing  (up arrow) three times
will return to the ruby command.

Answered By: jordanm

To add to @jordanm reply, I think you should use HISTCONTROL in fact but with the “erasedups” value.

“A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved.”

export HISTCONTROL="ignoreboth:erasedups"

Add it to your ~/.bashrc to have it execute every time you log in.

Actually, this has been answered before

I wanted to add a comment but I can’t because of my reputation.

Answered By: pablosan

For macos, add the following lines to your ~/.zshrc file to avoid command duplicates in terminal history

setopt HIST_EXPIRE_DUPS_FIRST
setopt HIST_IGNORE_DUPS
setopt HIST_IGNORE_ALL_DUPS
setopt HIST_IGNORE_SPACE
setopt HIST_FIND_NO_DUPS
setopt HIST_SAVE_NO_DUPS
Answered By: VVK

I really couldn’t find anything that works!

I create a bash file which I run on boot, but you can schedule it with cron too. It erase the duplicated from .bash_history.

arrVar=()
while read -r line; do
  s=${line# } # remove leading spaces
  s=${line%% } # remove trailing spaces
  if [[ ! " ${arrVar[*]} " =~ " ${s} " ]] && [[ "$s" != history* ]] ;
  then
    arrVar+=("$s")
  fi
done < ~/.bash_history

printf "%sn" "${arrVar[@]}" > ~/.bash_history
Answered By: Yoram
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.