01/10/08

Summary
- SUMS test: can't reproduce ingest problem.
- Doxygen search tool, 
- Added dontwait option for drms_stage_records()
- Checkpointing: drms_server_start_transaction(), drms_server_end_transaction()
- PostgresSQL compile from source

SUMS test
---------

~karen/src/karen_test/sum_put.c 
use SUM_alloc() to allocate 500 SU's, each 32MB, then store them with SUM_put().

In my test, I ran 10, 100 (took 1.5 hours to finish) jobs currently.
I was not able to reproduce the problem we saw with Hao's ingest
programs.

Jim noticed something wrong in the sums tables relating to tests. I
don't know whether he has resolved them.

Doxygen search tool 
-------------------
bug: search results contain wrong links to example files.
temporary fix: 
after running doxygen to generate html files, created symbolic links
from *-example.html to *-source.html.

gendox.csh is now run daily at 19:30.

drms_stage_records(): dontwait option
-------------------------------------

Commited code for drms_stage_records() including dontwait option which
does not wait for SUMS reply. "Dontwait" is currently implemented as
do not check sum_outbox for SUMS reply.

dontwait option exposed two problems in DRMS:

1. The sums inbox and outbox are fifo queues, i.e., dequeue only looks
at the head element of the queue. This has some implications. For
example, if a reply is not picked up from the front of queue, no
subsequent replied may ever be picked up. Since dontwait case does not
wait for SUMS reply, the reply should never be entered into the
outbox. (Otherwise the server thread will be waiting forever for reply
to commit log SU while the client thread left a reply in the outbox
not yet picked up before exiting).

2. in drms_storageunit.c local variable request in drms_storageunit.c
is a problem: when the server thread (that runs drm_su_getsudirs)
finishes, the request variable, being local to that function, becomes
garbage. And sums thread picks up this garbage request from the
inbox. This is not a problem if we wait until the request is served,
i.e., the sums thread uses the request variable before the function
exits.

Checkpointing
-------------

First cut implementation to commit both drms and sums in the middle of
a direct connect module.

Requirement:
- new session, new log

discovered more confusion in DRMS terminology:
- DRMS session is different from DB session.
  DRMS session contains exactly one transaction.
  DB session corresponds to a DB connection, may contain more than one
  transactions  

Added two new functions:
drms_server_start_transaction()
drms_server_end_transaction()

Many parameters shared among transactions, e.g., sessionns, dbhost, etc
- added more fields in DRMS_Env_t struct.

end_transaction tears down all all, except for the data db connnection and
the signal thread

- added 'final' flag to drms_free_env() and drms_server_comit() to
  differentiate bewteen checkpointing and exiting.

Met trouble in teeing/redirection...

PostgreSQL
----------

Compiled from source.

Found out how to compile it to run from a different port.

./configure --with-pgport=5434
The default port number is 5432