Cron, python and saving to a file

Question

Cron, python and saving to a file

I’ve got a veeery simple script written in python that tests my internet connection na saves parsed data to a json file:

#!/usr/bin/python3
import subprocess, json, os
from datetime import datetime

# run the bash script and parse results
def speedtest():
    result = subprocess.run(["/bin/speedtest-cli", "--simple"], stdout=subprocess.PIPE, encoding="UTF-8").stdout
    # result = os.system("speedtest-cli --simple")
    download_str = result.find("Download: ") + 10
    download_speed = result[download_str:result.find(" Mbit/s", download_str)]
    upload_str = result.find("Upload: ") + 8
    upload_speed = result[upload_str:result.find(" Mbit/s", upload_str)]
    ping_str = result.find("Ping: ") + 6
    ping = result[ping_str:result.find(" ms", ping_str)]
    return download_speed, upload_speed, ping

def save_to_json(data):
    # load existing data to a dict
    existing_data = []
    with open("/home/trendkiller/development/python/py_speedtest/data.json") as fp:
        existing_data = json.load(fp)
   
    payload = {
        "date": datetime.now().strftime("%d/%m/%Y"),
        "time": datetime.now().strftime("%H:%M"),
        "download": data[0] + " Mbps",
        "upload": data[1] + " Mbps",
        "ping": data[2] + " ms"
    }
    
    existing_data.append(payload)

    # save to a json file
    with open("/home/trendkiller/development/python/py_speedtest/data.json", "w") as f:
        json.dump(existing_data, f, indent=4)

def main():
    test_result = speedtest()
    # save to a json_file
    save_to_json(test_result)
    return 0

if __name__ == "__main__":
    main()

It runs smoothly when I call it from terminal.
I would like to automate it so it can test my connection hourly.
For this I’m trying to use CRON and it works when I schedule it to run every minute (data is saved to data.json).
But when I try to run it hourly, data isn’t passed to a json file.
Here’s my crontab file:

@hourly /bin/python3 /home/trendkiller/python/py_speedtest/main.py

While looking at the grep "CRON" /var/log/syslog
there’s nothing strange (to my knowledge):

Feb  5 08:00:01 home-lab CRON[119326]: (trendkiller) CMD (/bin/python3 /home/trendkiller/development/python/py_speedtest/main.py > /home/trendkiller/development/python/py_speedtest/log.txt)
Feb  5 08:00:04 home-lab CRON[119325]: (CRON) info (No MTA installed, discarding output)
Feb  5 08:17:01 home-lab CRON[119719]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Feb  5 09:00:01 home-lab CRON[120590]: (trendkiller) CMD (/bin/python3 /home/trendkiller/development/python/py_speedtest/main.py > /home/trendkiller/development/python/py_speedtest/log.txt)
Feb  5 09:00:02 home-lab CRON[120589]: (CRON) info (No MTA installed, discarding output)
Feb  5 09:17:01 home-lab CRON[120798]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

Any help would be very useful,
thanks in advance (and don’t mind the clutterred code :P).

Asked By: Trendkiller

||

Source

Answer 1

A quick answer:

Write a script in /etc/cron.hourly.

runpy script:

#!/bin/bash
/bin/python3 /home/trendkiller/python/py_speedtest/main.py

Then make it executable:

sudo chmod +x /etc/cron.hourly/runpy

Test with --report or --test:

run-parts --report /etc/cron.hourly

manpages.

Answered By: GAD3R

Answer 2

First of all, make the program executable (chmod u+x /home/trendkiller/python/py_speedtest/main.py) and ensure it has a valid Python interpreter as its first line. The cron command would suggest #!/bin/python3 but you’ve written #!/usr/bin/python3. Which is correct?

The CRON log output also suggests that the command you’ve shown us isn’t actually the command you’re running (one has a redirection of stdout to a log file, the other does not). Both are missing redirection of stderr.

You can either capture the output in the system logger:

Capture the output from your program, including errors written to stderr:

@hourly /home/trendkiller/python/py_speedtest/main.py 2>&1 | logger -t py_speedtest -p user.info

Read the logfile (probably grep py_speedtest /var/log/user.log)

Or if you’re not running a syslogger such as rsyslog you may prefer to just write the output to a file. But remember to truncate the file every so often, or replace it (use > instead of >>) each time your program runs rather than append to it.

Capture the output from your program, including errors written to stderr:

@hourly ( date; /home/trendkiller/python/py_speedtest/main.py )>"$HOME/.py_speedtest.log" 2>&1

Read the logfile (for example less "$HOME/.py_speedtest.log")

I would think it most likely that you’re not in the directory you think you’re in. If this is the case you can fix it by changing to your $HOME directory. For example,

@hourly cd && ( date; /home/trendkiller/python/py_speedtest/main.py )>".py_speedtest.log" 2>&1

(If your home directory is /home/trendkiller you can remove that from the command too: python/py_speedtest/main.py)

Answered By: Chris Davies