09/06/07 ---------------------------------------------------------------------- Make target changes: - Moved helloworld into target 'examples', since it requires third-party libraries. It's no longer part of the default targets. - Created a new target "sums" and removed from default targets, made it part of target "universe". This way the sums executables, e.g., sum_svc, tape_svc, etc, not part of the default targets. - I'd like to suggest moving targets in 'datacapture' and 'lev0' subdirs out of the default targets as well. Instead, we can do something similiar as that for 'sum' subdir, i.e., to create a seperate target lists for all executables/libraries in that subdir. ---------------------------------------------------------------------- Retention time for data segments reading from SUMS - In drms_su_getsudir(), set retention time to session env retention time if the latter is set. ---------------------------------------------------------------------- Fixed wal archive problem for db server on hmidb LOG: archive command "test ! -f /var/lib/pgsql/backups/wal_arch/000000010000019 4000000FE.gz && gzip -1 -c pg_xlog/0000000100000194000000FE > /var/lib/pgsql/backups/wal_arch/0000000100000194000000FE.gz" failed: return code 256 - This was a leftover problem from last month when I filled up the db disk with my testing data. - Restarting the db server did not fix the problem - Looks like we hit a bug in 8.1.2. I removed the troubled backup file /var/lib/pgsql/backups/wal_arch/0000000100000194000000FE.gz, and the error message went away. The archiving seems to proceed normally. This bug has been fixed in version 8.1.5. For more information, see http://archives.postgresql.org/pgsql-admin/2006-11/msg00051.php ---------------------------------------------------------------------- DB query performance test on hmidb2: DB server log: LOG: statistics buffer is full - solution: set stats_command_string to off. DB server log: LOG: checkpoints are occurring too frequently (20 seconds apart) HINT: Consider increasing the configuration parameter "checkpoint_segments". - solution: set checkpoint_segment to 2049, was 3 more than 150M records in one table, each column fully populated. All columns, except recnum, fsn, and t_obs, are the same as that of hmi_ground.lev0 recnum=1000. ---------------------------------------------------------------------- Our original design and implementation make a record read-only after it is closed. Scenarios arise where subsequent pipeline modules would want to update the same record, perhaps supplying a different set of keywords. Phil deemed it reasonable to relax such read-only restriction and to allow more than one modules to modify a record as long as they are within the same session. - Implementation: in drms_close_records(), don't even check the read-only flag if a record's session information is the same as that of the current session. - Drawback: API change . method I : add a DRMS_Env_t *env to drms_close_records() . method II : remove all DRMS_Env_t *env from all APIs. The drms env variable is a global variable. It's given a different name in drms/drms_server.c and libdrms/jsoc_main_c.c, env in the former, and drms_env the latter. If we give them the same name, none of the API functions will need this parameter. There are more than 60 such functions that use DRMS_Env_t *env as an input parameter. I don't see a clear win for either method, as both involve updating existing modules. - Problem to solve: update cannot use the same drms_insert_records() which uses SQL COPY on the same recnum. It fails because it violates unique constraint on recnum. What we need is a new drms_update_records() function which uses SQL UPDATE ---------------------------------------------------------------------- In progress: - DB query performance test - To commit code for direct drms connection. It's time consuming to merge code... - drms_update_records()