TIL 1:
Incrementing times with the date
command
- I have been working on an HPR project which is restoring external
files to shows which lost them when we migrated to the current static
site.
- As I make changes I want to be able to check that they are correct,
but to make this check I need to wait for the next update of the static
site.
- Ken was away recently, and set up a
cron
job to refresh
the site every three hours. Each show page shows the refresh time in UTC
form in the header.
- Being a bit numerically challenged I wanted a way of computing the
next refresh time in my timezone from the previous refresh time.
- The GNU
date
command accepts a date and time expression
after the -d
option (or using the
alternative--date=STRING
option). The contents of the
STRING
here are very flexible but quite complex since you
can include time zone data, offsets, day and month names, etc. See the
links below for links to the GNU manual.
- My first attempt used the
date
command like this and
got the wrong answer (using the output format +%T
which
writes the time in a default form):
$ date -d '16:27:16 + 3 hours' +%T
15:27:16
- It is not clear why this fails, but the GNU function which parses
these date parameters is obviously confused. The second try included the
time zone after the time, and worked better, but is a little
confusing:
$ date -d '16:27:16 UTC + 3 hours' +%T
20:27:16
- The time returned is local time for me. The
date
command has added three hours to the UTC date to get 19:27:16, but since
I am in the UK, which is in DST (called BST - British Summer Time - UTC
plus 1 hour), an hour is added.
- The final try used the
-u
option which writes UTC
time:
$ date -u -d '16:27:16 UTC + 3 hours' +%T
19:27:16
- I actually ended up using and re-using these commands (though a
script would have been better):
$ current='06:27:55'
$ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
09:30:55 UTC / 10:30:55 BST
$ current=$next
$ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
12:33:55 UTC / 13:33:55 BST
$ current=$next
$ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
15:36:55 UTC / 16:36:55 BST
TIL 1: Links
TIL 2: Merging lines of
files with paste
- While processing and "repairing" shows I came across the need to
generate a list of show numbers separated by commas. In the past I have
loaded these into a Bash array and turned them into a comma-delimited
string, using the parameter substitution capabilities of Bash which can
add a comma to each element. The trouble with this is that it leaves a
trailing comma which has to be removed. I stumbled upon
paste
as an alternative way of doing this.
- The GNU
paste
command is another from the GNU
Coreutils group. This one merges lines of files. Its synopsis
is:
paste [OPTION]... [FILE]...
- It merges lines consisting of the corresponding lines from each file
provided as an argument, by default separated by TABs, and writes them
to standard output. This means it produces lines consisting of the first
line from each of the files, separated by tabs, then the second lines,
and so on.
- Any of the files can be linked to standard in by using a
-
(hyphen) as the file name.
- The delimiters can be changed with the
-d LIST
or
--delimiters=LIST
option. The use of a list of delimiters
causes the characters in the list to be used sequentially for each
delimiter.
- The merged lines can be visualised as rows in a matrix, where each
file provides a column.
- The "matrix" is rotated by using the
-s
or
--serial
option. Here the lines from one file at a time are
merged - the files are processed serially rather than in
parallel.
- The
paste
command can be used to generate the
comma-delimited list I wanted by using the options -s
and
-d ','
:
$ printf '%s&bsoln' {1..10} | paste -s -d, -
1,2,3,4,5,6,7,8,9,10
- Note that you can't get the same result with
echo {1..10}
because all the numbers will be written to one
line rather than being the separate lines that paste
requires.
- The file arguments to
paste
may also be Bash
process substitution expressions:
$ paste -s -d, <(printf '%s&bsoln' {1..10})
1,2,3,4,5,6,7,8,9,10
- This means you can generate more complex output by using multiple
process substitution expressions where each is seen as a
file:
$ paste -d'|' <(printf '%d&bsoln' {1..7}) <(printf '%s&bsoln' {A..G}) <(printf '%d&bsoln' {100..106})
1|A|100
2|B|101
3|C|102
4|D|103
5|E|104
6|F|105
7|G|106
- Note how using the
-s
option "rotates" this:
$ paste -s -d'|' <(printf '%d&bsoln' {1..7}) <(printf '%s&bsoln' {A..G}) <(printf '%d&bsoln' {100..106})
1|2|3|4|5|6|7
A|B|C|D|E|F|G
100|101|102|103|104|105|106
- This is what I was actually trying to do, so I could feed show
numbers to another script which will accept a CSV list as an
option:
$ echo "select episode_id from repairs where repair_date is null order by episode_id desc limit 5" |&bsol
sqlite3 -list ~/HPR/InternetArchive/ia.db | paste -s -d',' -
1959,1952,1951,1946,1941
TIL 2: Links