Production table

To create the file for updating the productions:

   psql -U postgres -W -h db.pau.pic.es dm -c "COPY (SELECT * FROM production) TO stdout DELIMITER ',' CSV" | pv  > paudm_production_datetime.csv

To remove the old version:

   hdfs dfs -rm /user/jcarrete/data/paudm/production/*

Copy from local:

   hdfs dfs -copyFromLocal /cephfs/pic.es/astro/scratch/jcarrete/sandbox/paudm_production_datetime.csv  /user/jcarrete/data/paudm/production/paudm_production_datetime.csv

To update the table:

   INSERT OVERWRITE TABLE cosmohub.paudm_production SELECT id, input_production, pipeline, release, software_version, job_id, comments, created FROM jcarrete.paudm_production_csv;

There were problems with the "commment" fields. This was reported in 14/09/2020. Some comments (8 entries) contained "line break" and then the number of entries in the csv file were wrong. We asked for modifying them directly in the paudb given that there are only 8 entries.