20160928

When key_my_trc_iceeco is activated, the run stops here (ocean.output):

 trcdit_wr : write NetCDF passive tracer concentrations at            3 time-step
 ~~~~~~~~~

But when I submit the job with "#PBS -l nodes=1:ppn=12" as opposed to proc=12, then the run went one step further:

 trcdit_wr : write NetCDF passive tracer concentrations at            3 time-step
 ~~~~~~~~~
 trcdii_wr : write NetCDF additional arrays at            3 time-step
 ~~~~~~

Actually, I repeated the run with "#PBS -l nodes=1:ppn=12", but this time it did not reproduce the same result as before, but the same result with 'procs=12' case... very strange.



But still being killed due to 174 seg fault and 78 error (not sure what they mean):

forrtl: severe (174): SIGSEGV, segmentation fault occurred

forrtl: error (78): process killed (SIGTERM)

It seems that this happens at the third time step (I tested with "write" statements).

Maybe I need to ask for more procs? Try with procs=24:

The run went bit further.

Now increase vmem=3000mb:

No further progress.. So I don't think it's the lack of memory/processors.

I will look into MY_TRC again..