Files in the top-level directory from the latest check-in
- Amy.jl
- array.sh
- B2funs.jl
- BayOfFundy.jl
- c2noise.jl
- checkdat.jl
- cleandat.jl
- cleanout.jl
- copyrads.sh
- DAops.jl
- DAsymmetrychecks.jl
- datahandlers.jl
- diagnostics.jl
- extras.jl
- fftest.jl
- fftest_a.sh
- fftest_a_allcpu.sh
- fftest_bench.jl
- fftest_cleanup.jl
- fftest_cleanup.sh
- fftest_cleanup.sh.template
- fftest_ha.jl
- fftest_one.sh
- fftest_par.jl
- fftest_tmp.jl
- fftest_uv.jl
- fftest_x.sh
- fftest_x1.sh
- fftest_x2.sh
- fftest_x3.sh
- fftest_x4.sh
- fftest_x_driver.sh
- fftest_y.sh.template
- fftest_y_driver.sh
- fftest_z.sh.template
- fftest_z_driver.sh
- fftwtest.jl
- genHA.jl
- genHA_a.sh
- genSSH.jl
- genSSH_a.sh
- genUV.jl
- genUV_a.sh
- geography.jl
- getexpl.jl
- gnom.jl
- HofX.jl
- JimThomas.jl
- job001.sh
- kcompare.jl
- lars2x.jl
- lars2z.jl
- load_balance.jl
- partition.jl
- partitionlook.jl
- partitionsearch.jl
- patchmerge.jl
- patchmerge.sh
- patchmerge_special.jl
- patchone.jl
- psd2d.jl
- psd2d_nslcst.jl
- psdfuncs.jl
- psdsolver.jl
- README
- README-c2noise
- savitskygolay.jl
- Step0.jl
- Step1.jl
- Step2.jl
- Step3.jl
- submit001.sh
- summary.jl
- tmp.jl
- uv.jl
Sequence for a clean run:
mv scratch/workdir scratch/workdirOLDVERSION
# Choose L, set machine-dependent parameters related to
# the domain decomposition and load balancing.
# Check that every directory contains a lonlatL.nc file.
julia load_balance.jl
# Partition the nodes into disjoint "colors" for more efficient
# processing in parallel. Only needs to be redone if L or padding
# is changed; otherwise, it can be copied.
julia partition.jl
# Choose constituents to be analyzed. Edit genHA.jl and run
# genHA_a.sh. Generate HA.nc in every directory.
# Check if the jobs are spawned:
# squeue -u ezaron
# Check the files in ./slurmout for errors.
# Check if HA.nc files exist and look reasonable.
# Edit checkdat.jl and run for graphical map.
rm ./slurmout/*
sbatch genHA_a.sh
tail slurmout/*_err.txt
-- many failing due to time! Randomly permute the directories to load from.
-- try again on the medium queue (4 days!).
sbatch genSSH_a.sh
sbatch genUV_a.sh
-- many fail due to memory usage. Other than reducing procs, I don't see an
easy workaround. Run it 4 or 5 times.
Run the solver:
Original driver: this will use the entire machine:
./fftest_x_driver.sh
---> Hmmm. A small, but non-negligible, number of UV files are ill-formed, missing LON!
Run cleandat.jl to remove incomplete files. And rerun genUV.jl.
New driver: this will monitor the queue and try to leave some spare nodes for others:
./fftest_y_driver.sh
NOTE IN LATEST GOOD RUN: I hardwired Lplane to be twice the prescribed size!
DON'T FORGET TO USE "CLEAN" OPTION WHEN RE-RUNNING fftest.jl WITH A DIFFERENT TIDE.
Aha! Need to regenerate everything for K1 and O1. The fitting window needs to be at
least twice as large!
---> Even with large patches, the solution based on HA is no good! Need to debug it in detail. FOUND THE BUG -- it was my bilinear interpolation!
And fixed it to make the 11.3 run.
2022-08-25:
# Is the validation data (explained variance) significantly changed if
I use IB instead of the full dac or ECMWF Interim instead of nominal
WPD? Or if I use the new DT-2021 mesoscale correction?
Note that for this MARN run, the validation data is just c2a and the Jason-GMs.
NOMINAL MARN Case:
Explained variance table:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.36 & 0.39 & 14.8 & 10.9 & 2.9
resx & 0.30 & 0.44 & 28.6 & 9.9 & 4.0
--------------------------------------------------------------
res11 & 0.37 & 0.37 & 12.8 & 13.4 & 3.1
res12 & 0.37 & 0.37 & 12.8 & 13.4 & 3.1
res13 & 0.37 & 0.37 & 12.8 & 13.4 & 3.1
--------------------------------------------------------------
res21 & 0.02 & 0.03 & 1.5 & 1.1 & 1.0
res22 & 0.03 & 0.03 & 1.6 & 1.2 & 1.0
res23 & 0.02 & 0.03 & 1.5 & 1.1 & 1.0
--------------------------------------------------------------
res31 & 0.30 & 0.44 & 28.6 & 9.9 & 4.0
res32 & 0.27 & 0.42 & 30.5 & 9.4 & 4.1
res33 & 0.24 & 0.40 & 31.3 & 9.0 & 4.1
--------------------------------------------------------------
From patchone.jl
GM Summary stats:
model & GM [cm^2] & GMs [mm^2] \\
resx & 0.44 & 4.0 \\
res11 & 0.37 & 3.1 \\
res22 & 0.03 & 1.0 \\
res33 & 0.40 & 4.1 \\
AGREES!
USING LatestMesoscale MARN Case:
GM Summary stats:
model & GM [cm^2] & GMs [mm^2] \\
resx & 0.56 & 4.9 \\
res11 & 0.42 & 3.7 \\
res22 & 0.04 & 1.1 \\
res33 & 0.53 & 5.0 \\
DOES NOT AGREE EXACTLY, BUT THE RELATIVE RANKING IS RETAINED!
USING IB INSTEAD OF DAC MARN Case:
GM Summary stats:
model & GM [cm^2] & GMs [mm^2] \\
resx & 0.56 & 4.9 \\
res11 & 0.42 & 3.7 \\
res22 & 0.04 & 1.1 \\
res33 & 0.53 & 5.0 \\
USING ECMWF Interim instead of nominal WPD:
GM Summary stats:
model & GM [cm^2] & GMs [mm^2] \\
resx & 0.51 & 4.5 \\
res11 & 0.41 & 3.6 \\
res22 & 0.04 & 1.1 \\
res33 & 0.47 & 4.6 \\
Hmmm. The fact that these all changed, compared to the original, suggests
that either unrelated changes in the data cause these differences, or
the biggest change is related to the Mesoscale Correction.
Based on what I can see in the model/data misfits, the main issue here
is a deficiency in the altimetry along some of the tracks, nothing
particularly special about the drifter data.
# Get better locations, centered on my plots from tai:Desktop/NSF-Drifters/Julia/hret113.jl
From coeus:FFTest/parititionsearch.jl:
MARN: GDP is a little better
The job closest to (331.000000 lon, 33.000000 lat) is found at:
workdir = /scratch/ezaron/workdir/n0114/p011/j002/
offshore Amazon: HRET is much better
The job closest to (317.000000 lon, 11.000000 lat) is found at:
workdir = /scratch/ezaron/workdir/n0092/p003/j002/
Tuomoto: GDP is a lot better
The job closest to (210.000000 lon, -13.000000 lat) is found at:
workdir = /scratch/ezaron/workdir/n0059/p018/j002/
Indo: no GDP and HA is much worse, too. Why?
The job closest to (127.000000 lon, -2.000000 lat) is found at:
workdir = /scratch/ezaron/workdir/n0072/p014/j006/
Orig solution stats (old outlier criterion):
Explained variance table:
model & dof & HA [cm^2] & GM [cm^2] & UV [cm^2/s^2] & HAs [mm^2] & GMs [mm^2] \\
--------------------------------------------------------------
HRET & 216 & 4.73 & 6.46 & -6.0 & 158.4 & 148.4 \\
resx & 0 & 0.00 & -0.00 & 1.8 & 0.1 & -0.1 \\
-- models built from HA data ---------------------------------
number of HA data = 1126
res11 & 174 & 5.28 & 5.38 & -36.9 & 208.2 & 125.7 \\
res12 & 168 & 5.26 & 5.36 & -35.6 & 207.2 & 125.9 \\
res13 & 2 & 0.30 & 0.27 & -2.6 & 10.9 & 7.6 \\
-- models built from GM data ---------------------------------
number of GM data = 13357
res21 & 90 & 1.43 & 2.54 & 1.6 & 41.9 & 184.3 \\
res22 & 91 & 1.44 & 2.57 & 1.5 & 42.2 & 186.5 \\
res23 & 82 & 1.39 & 2.46 & 1.6 & 40.9 & 178.0 \\
-- models built from UV data ---------------------------------
number of UV data = 1392
res31 & 3 & 0.00 & -0.00 & 1.8 & 0.1 & -0.1 \\
res32 & 21 & -0.06 & -0.04 & 62.6 & -8.4 & 3.2 \\
res33 & 106 & -3.44 & -3.26 & 227.6 & -262.4 & -310.0 \\
--------------------------------------------------------------
New solution stats (with new outlier criterion):
Explained variance table:
model & dof & HA [cm^2] & GM [cm^2] & UV [cm^2/s^2] & HAs [mm^2] & GMs [mm^2] \\
--------------------------------------------------------------
HRET & 216 & 6.19 & 6.46 & -6.0 & 203.2 & 148.4 \\
resx & 0 & 0.13 & 0.11 & 3.6 & 5.9 & 2.9 \\
-- models built from HA data ---------------------------------
number of HA data = 1765
res11 & 224 & 6.55 & 5.40 & 13.7 & 303.7 & 147.3 \\
res12 & 223 & 6.51 & 5.38 & 15.5 & 302.1 & 146.8 \\
res13 & 80 & 3.69 & 3.29 & 50.0 & 187.7 & 92.0 \\
-- models built from GM data ---------------------------------
number of GM data = 13357
res21 & 112 & 1.80 & 2.88 & 0.7 & 55.6 & 213.5 \\
res22 & 113 & 1.80 & 2.89 & 0.7 & 55.7 & 214.6 \\
res23 & 82 & 1.59 & 2.46 & 1.6 & 50.7 & 178.0 \\
-- models built from UV data ---------------------------------
number of UV data = 1392
res31 & 2 & -0.00 & -0.00 & 1.6 & -0.1 & -0.2 \\
res32 & 21 & -0.09 & -0.04 & 62.6 & -10.0 & 3.2 \\
res33 & 108 & -3.74 & -4.03 & 233.1 & -321.4 & -384.7 \\
--------------------------------------------------------------
====> Note how the res11 solution fits HA, and especially HAs and GMs much better than
in the original.
Does the size of Lplace influence this case? Try using the nominal Lplane.
Explained variance table:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 6.19 & 6.46 & -6.0 & 203.2 & 148.4
resx & 6.42 & 5.47 & 29.1 & 294.9 & 147.9
--------------------------------------------------------------
res11 & 6.43 & 5.48 & 28.9 & 295.2 & 148.0
res12 & 6.42 & 5.47 & 29.1 & 294.9 & 147.9
res13 & 3.62 & 3.34 & 46.4 & 181.8 & 88.5
--------------------------------------------------------------
res21 & 1.89 & 2.80 & -2.0 & 59.2 & 201.4
res22 & 1.89 & 2.82 & -2.1 & 59.4 & 202.4
res23 & 0.01 & 0.01 & 0.7 & 0.4 & 3.3
--------------------------------------------------------------
res31 & -0.00 & -0.00 & 1.9 & -0.4 & -0.1
res32 & -0.05 & -0.02 & 57.9 & -8.7 & 5.7
res33 & -2.42 & -2.42 & 227.8 & -159.2 & -169.1
--------------------------------------------------------------
==> Some slight differences, but not sure what to make of it!
Also, note that I fixed resx. It is now computed as best the linear combination
of res12 and res32, constrainted to lie between them.
offshore Amazon: HRET is much better
The job closest to (317.000000 lon, 11.000000 lat) is found at:
workdir = /scratch/ezaron/workdir/n0092/p003/j002/
Original: (2 Lplane)Explained variance table:
model & dof & HA [cm^2] & GM [cm^2] & UV [cm^2/s^2] & HAs [mm^2] & GMs [mm^2] \\
--------------------------------------------------------------
HRET & 216 & 0.82 & 0.75 & 12.6 & 27.7 & 14.9 \\
resx & 0 & 0.58 & 0.59 & 22.5 & 19.3 & 11.2 \\
-- models built from HA data ---------------------------------
number of HA data = 2859
res11 & 205 & 0.85 & 0.78 & 12.3 & 30.5 & 16.2 \\
res12 & 204 & 0.85 & 0.78 & 12.3 & 30.5 & 16.2 \\
res13 & 181 & 0.83 & 0.77 & 12.5 & 30.0 & 16.0 \\
-- models built from GM data ---------------------------------
number of GM data = 92266
res21 & 84 & 0.56 & 0.60 & 10.0 & 20.8 & 13.9 \\
res22 & 85 & 0.56 & 0.60 & 10.0 & 21.0 & 14.1 \\
res23 & 84 & 0.56 & 0.60 & 10.0 & 20.8 & 13.9 \\
-- models built from UV data ---------------------------------
number of UV data = 90042
res31 & 265 & 0.58 & 0.58 & 22.6 & 19.1 & 11.1 \\
res32 & 257 & 0.58 & 0.58 & 22.3 & 19.1 & 11.1 \\
res33 & 414 & 0.53 & 0.55 & 25.6 & 17.6 & 10.5 \\
--------------------------------------------------------------
New (1 Lplane and fixed outliers and such):
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.84 & 0.75 & 12.6 & 28.6 & 14.9
resx & 0.84 & 0.77 & 14.7 & 30.5 & 16.2
--------------------------------------------------------------
res11 & 0.85 & 0.77 & 12.4 & 31.0 & 16.3
res12 & 0.85 & 0.77 & 12.4 & 31.0 & 16.3
res13 & 0.84 & 0.76 & 12.4 & 30.7 & 16.1
--------------------------------------------------------------
res21 & 0.55 & 0.57 & 9.6 & 20.7 & 13.4
res22 & 0.55 & 0.58 & 9.7 & 20.9 & 13.6
res23 & 0.55 & 0.57 & 9.6 & 20.7 & 13.4
--------------------------------------------------------------
res31 & 0.61 & 0.60 & 22.3 & 20.2 & 11.2
res32 & 0.61 & 0.60 & 22.3 & 20.2 & 11.2
res33 & 0.57 & 0.58 & 25.1 & 18.9 & 10.8
--------------------------------------------------------------
Far N near Iceland:
"../workdir/n0128/p011/j003/" # Far N near Iceland
Explained variance table:
model & dof & HA [cm^2] & GM [cm^2] & UV [cm^2/s^2] & HAs [mm^2] & GMs [mm^2] \\
--------------------------------------------------------------
HRET & 216 & 0.03 & 0.03 & -18.4 & 1.1 & 0.2 \\
resx & 0 & 0.01 & 0.02 & 106.7 & 0.6 & -0.1 \\
-- models built from HA data ---------------------------------
number of HA data = 7274
res11 & 442 & 0.11 & 0.10 & -246.0 & 10.6 & 0.9 \\
res12 & 307 & 0.09 & 0.08 & -136.2 & 8.0 & 1.0 \\
res13 & 2 & 0.00 & 0.00 & -0.2 & 0.1 & 0.0 \\
-- models built from GM data ---------------------------------
number of GM data = 107098
res21 & 8 & 0.00 & 0.00 & -2.0 & 0.0 & 0.3 \\
res22 & 101 & -0.00 & 0.02 & -61.5 & -0.3 & 3.8 \\
res23 & 2 & 0.00 & 0.00 & -0.3 & 0.0 & 0.0 \\
-- models built from UV data ---------------------------------
number of UV data = 661356
res31 & 602 & 0.01 & 0.02 & 106.7 & 0.6 & -0.1 \\
res32 & 180 & 0.01 & 0.02 & 50.0 & 0.4 & 0.1 \\
res33 & 1046 & -0.21 & -0.18 & 193.6 & -4.8 & -2.2 \\
--------------------------------------------------------------
New
Explained variance table:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.03 & 0.04 & -16.6 & 1.0 & 0.2
resx & 0.01 & 0.01 & 29.5 & 0.4 & 0.0
--------------------------------------------------------------
res11 & 0.33 & 0.04 & -665.2 & 32.3 & -1.5
res12 & 0.00 & -0.00 & 0.3 & 0.3 & -0.1
res13 & 0.00 & -0.00 & 0.3 & 0.3 & -0.1
--------------------------------------------------------------
res21 & 0.00 & 0.01 & -23.6 & 0.1 & 1.8
res22 & 0.00 & 0.02 & -57.2 & 0.0 & 3.5
res23 & 0.00 & 0.00 & -0.6 & 0.0 & 0.1
--------------------------------------------------------------
res31 & 0.03 & 0.02 & 145.0 & 1.4 & -0.4
res32 & 0.01 & 0.01 & 29.5 & 0.4 & 0.0
res33 & 0.02 & 0.01 & 157.7 & 1.4 & -0.5
--------------------------------------------------------------
==> Note the very large number of dof identified. And there are a lot of
drifter data in the domain. The currents seem too large to be barotropic
currents. Can I map the currents and see if they are localized near topo
or something?
######### Minimum depth exclusion for GM SSH:
Tuomoto SE:
Original:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 1.13 & 1.28 & 11.8 & 29.0 & 12.1
resx & 1.24 & 1.41 & 30.2 & 42.7 & 15.1
--------------------------------------------------------------
res11 & 1.61 & 0.77 & -26.4 & 87.4 & 9.5
res12 & 1.61 & 0.77 & -26.1 & 87.0 & 9.5
res13 & 0.00 & -0.00 & -0.1 & 0.9 & -0.0
--------------------------------------------------------------
res21 & 0.86 & 0.97 & 9.0 & 29.0 & 16.5
res22 & 0.87 & 0.98 & 9.0 & 29.1 & 16.5
res23 & 0.86 & 0.97 & 9.0 & 29.0 & 16.5
--------------------------------------------------------------
res31 & 0.85 & 1.32 & 33.5 & 19.1 & 11.3
res32 & 0.86 & 1.34 & 35.4 & 18.5 & 11.6
res33 & 0.86 & 1.34 & 35.4 & 18.5 & 11.6
--------------------------------------------------------------
New: (fresh GM SSH with depth constraint)
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 1.13 & 1.27 & 11.8 & 29.0 & 11.1
resx & 1.00 & 1.38 & 33.6 & 27.8 & 12.4
--------------------------------------------------------------
res11 & 1.76 & 0.72 & -33.6 & 94.2 & 1.7
res12 & 1.21 & 0.59 & -11.9 & 67.3 & 2.9
res13 & 0.00 & -0.00 & -0.1 & 0.9 & -0.1
--------------------------------------------------------------
res21 & 0.67 & 0.89 & 10.5 & 17.1 & 12.0
res22 & 0.67 & 0.89 & 10.6 & 17.2 & 12.1
res23 & 0.67 & 0.89 & 10.5 & 17.1 & 12.0
--------------------------------------------------------------
res31 & 0.86 & 1.35 & 33.6 & 19.1 & 11.5
res32 & 0.86 & 1.36 & 35.2 & 18.6 & 11.7
res33 & 0.86 & 1.36 & 35.3 & 18.5 & 11.7
--------------------------------------------------------------
Again: modified the datahandlers: Note how res11 is fitting much better!
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 1.12 & 1.30 & 11.8 & 28.0 & 10.9
resx & 1.26 & 1.50 & 31.6 & 37.6 & 13.1
--------------------------------------------------------------
res11 & 1.50 & 1.07 & -6.0 & 69.8 & 7.4
res12 & 1.39 & 1.02 & -3.0 & 64.9 & 7.5
res13 & 0.72 & 0.64 & 4.8 & 32.7 & 5.7
--------------------------------------------------------------
res21 & 0.68 & 0.89 & 10.3 & 19.8 & 11.5
res22 & 0.68 & 0.90 & 10.4 & 20.0 & 11.6
res23 & 0.68 & 0.89 & 10.3 & 19.8 & 11.5
--------------------------------------------------------------
res31 & 1.01 & 1.44 & 34.6 & 21.0 & 11.5
res32 & 1.01 & 1.44 & 35.6 & 20.8 & 11.6
res33 & 1.01 & 1.44 & 35.6 & 20.8 & 11.6
--------------------------------------------------------------
Mid-Indian:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.3 & 0.0
resx & 0.05 & 0.05 & 0.7 & 0.7 & 0.0
--------------------------------------------------------------
res11 & 0.11 & 0.07 & 0.4 & 2.5 & -0.2
res12 & 0.05 & 0.04 & 0.4 & 0.7 & 0.0
res13 & 0.08 & 0.06 & 0.6 & 1.4 & -0.1
--------------------------------------------------------------
res21 & -0.00 & 0.00 & 0.0 & -0.0 & 0.0
res22 & -0.00 & 0.00 & 0.0 & -0.0 & 0.1
res23 & -0.00 & 0.00 & 0.0 & -0.0 & 0.0
--------------------------------------------------------------
res31 & 0.06 & 0.06 & 1.0 & 0.7 & -0.0
res32 & 0.05 & 0.05 & 0.7 & 0.7 & 0.0
res33 & 0.04 & 0.04 & 2.9 & 0.1 & -0.4
--------------------------------------------------------------
Mid-Indian, just 2 modes.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.10 & 0.07 & 0.5 & 1.2 & 0.0
resx & 0.06 & 0.06 & 0.9 & 0.7 & 0.0
--------------------------------------------------------------
res11 & 0.10 & 0.07 & 0.5 & 2.2 & -0.1
res12 & 0.05 & 0.05 & 0.5 & 0.7 & 0.0
res13 & 0.09 & 0.07 & 0.6 & 1.5 & -0.0
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.0 & 0.0
res22 & -0.00 & 0.00 & -0.0 & -0.0 & 0.0
res23 & -0.00 & 0.00 & -0.0 & -0.0 & 0.0
--------------------------------------------------------------
res31 & 0.06 & 0.06 & 1.1 & 0.7 & 0.0
res32 & 0.06 & 0.06 & 0.9 & 0.7 & 0.0
res33 & 0.03 & 0.03 & 2.6 & 0.0 & -0.2
--------------------------------------------------------------
Hmmm. HRET changed. That means it must be the modal sum.
Based on looking at the data, this seems to be a case where
the data window should be larger. Or gfo omitted!
Note: res31 and res32 are slightly better with only 2 modes.
Likewise, so are res12 and res13 and resx!
Mid-Indian, just 2 modes, but using full HRET:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.3 & 0.0
resx & 0.06 & 0.06 & 0.9 & 0.7 & 0.0
--------------------------------------------------------------
res11 & 0.10 & 0.07 & 0.5 & 2.2 & -0.1
res12 & 0.05 & 0.05 & 0.5 & 0.7 & 0.0
res13 & 0.09 & 0.07 & 0.6 & 1.5 & -0.0
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.0 & 0.0
res22 & -0.00 & 0.00 & -0.0 & -0.0 & 0.1
res23 & -0.00 & 0.00 & -0.0 & -0.0 & 0.0
--------------------------------------------------------------
res31 & 0.06 & 0.06 & 1.2 & 0.7 & 0.0
res32 & 0.06 & 0.06 & 0.9 & 0.7 & 0.0
res33 & 0.03 & 0.04 & 2.4 & 0.2 & -0.2
--------------------------------------------------------------
Mid-Indian, just 1 mode:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.3 & 0.04
resx & 0.06 & 0.07 & 1.0 & 0.9 & 0.02
--------------------------------------------------------------
res11 & 0.11 & 0.07 & 0.6 & 1.8 & -0.03
res12 & 0.07 & 0.05 & 0.6 & 1.0 & 0.03
res13 & 0.09 & 0.07 & 0.7 & 1.5 & 0.00
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.0 & 0.01
res22 & -0.00 & 0.00 & -0.0 & -0.0 & 0.03
res23 & -0.00 & 0.00 & -0.0 & -0.0 & 0.01
--------------------------------------------------------------
res31 & 0.06 & 0.07 & 1.2 & 0.9 & 0.00
res32 & 0.06 & 0.07 & 1.0 & 0.9 & 0.02
res33 & 0.02 & 0.03 & 2.1 & 0.1 & -0.22
--------------------------------------------------------------
Even better! Note additional sig fig on GMs.
Mid-Indian, just 1 mode, bw=0.23 instead of 0.45.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.3 & 0.04
resx & 0.08 & 0.08 & 0.9 & 1.2 & 0.04
--------------------------------------------------------------
res11 & 0.11 & 0.08 & 0.6 & 1.6 & 0.01
res12 & 0.10 & 0.07 & 0.7 & 1.3 & 0.04
res13 & 0.09 & 0.07 & 0.7 & 1.3 & 0.04
--------------------------------------------------------------
res21 & -0.00 & 0.00 & 0.0 & -0.1 & 0.05
res22 & -0.00 & 0.00 & 0.0 & -0.1 & 0.06
res23 & -0.00 & 0.00 & 0.0 & -0.1 & 0.05
--------------------------------------------------------------
res31 & 0.06 & 0.08 & 0.9 & 1.0 & 0.03
res32 & 0.06 & 0.08 & 0.9 & 1.0 & 0.03
res33 & 0.04 & 0.05 & 1.6 & 0.6 & -0.10
--------------------------------------------------------------
Note further improvements!
Mid-Indian, just 1 mode, bw=0.23, Lplane doubled:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.3 & 0.04
resx & 0.09 & 0.08 & 0.7 & 1.3 & 0.04
--------------------------------------------------------------
res11 & 0.11 & 0.09 & 0.7 & 1.7 & 0.02
res12 & 0.09 & 0.08 & 0.7 & 1.3 & 0.04
res13 & 0.09 & 0.08 & 0.7 & 1.3 & 0.04
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.1 & 0.05
res22 & -0.00 & 0.00 & -0.0 & -0.1 & 0.06
res23 & -0.00 & 0.00 & -0.0 & -0.1 & 0.05
--------------------------------------------------------------
res31 & 0.06 & 0.07 & 1.1 & 1.0 & 0.02
res32 & 0.06 & 0.07 & 1.0 & 0.9 & 0.02
res33 & 0.03 & 0.03 & 1.8 & 0.4 & -0.14
--------------------------------------------------------------
Note slight degradation of UV, but overall improvement.
As above, but omit GFO data:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.11 & 0.09 & 0.7 & 1.8 & 0.03
--------------------------------------------------------------
res11 & 0.13 & 0.10 & 0.6 & 2.3 & -0.01
res12 & 0.11 & 0.09 & 0.7 & 1.8 & 0.03
res13 & 0.11 & 0.09 & 0.7 & 1.8 & 0.03
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.1 & 0.05
res22 & -0.00 & 0.00 & -0.0 & -0.1 & 0.06
res23 & -0.00 & 0.00 & -0.0 & -0.1 & 0.05
--------------------------------------------------------------
res31 & 0.08 & 0.07 & 1.1 & 1.2 & 0.02
res32 & 0.08 & 0.07 & 1.0 & 1.2 & 0.02
res33 & 0.06 & 0.04 & 1.7 & 0.7 & -0.11
--------------------------------------------------------------
Note improvement wrt to GM and HAs, but not GMs
Mid-Indian, just 1 mode, bw=0.23, Lplane back to default (best previous
case so far), and exclude GFO:
BEST SO FAR:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.09 & 0.08 & 0.9 & 1.5 & 0.04
--------------------------------------------------------------
res11 & 0.13 & 0.09 & 0.6 & 2.2 & 0.00
res12 & 0.11 & 0.07 & 0.7 & 1.6 & 0.04
res13 & 0.11 & 0.07 & 0.7 & 1.7 & 0.04
--------------------------------------------------------------
res21 & -0.00 & 0.00 & 0.0 & -0.1 & 0.05
res22 & 0.00 & 0.00 & 0.0 & -0.1 & 0.06
res23 & -0.00 & 0.00 & 0.0 & -0.1 & 0.05
--------------------------------------------------------------
res31 & 0.08 & 0.08 & 0.9 & 1.3 & 0.03
res32 & 0.08 & 0.08 & 0.9 & 1.3 & 0.03
res33 & 0.06 & 0.05 & 1.6 & 0.8 & -0.09
--------------------------------------------------------------
Hard to compare since HA dataset is now different, but certainly
no degradation wrt GM.
Mid-Indian, 1 mode, bw=0.15, Lplane default, and exclude GFO:
slight degradation. Return to bw=0.23.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.10 & 0.08 & 0.7 & 1.4 & 0.04
--------------------------------------------------------------
res11 & 0.15 & 0.07 & 0.1 & 2.4 & -0.18
res12 & 0.11 & 0.05 & 0.4 & 1.5 & -0.01
res13 & 0.11 & 0.05 & 0.4 & 1.5 & -0.01
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.3 & 0.08
res22 & -0.01 & 0.00 & -0.1 & -0.5 & 0.11
res23 & -0.00 & 0.00 & -0.0 & -0.3 & 0.08
--------------------------------------------------------------
res31 & 0.08 & 0.08 & 0.7 & 1.1 & 0.03
res32 & 0.08 & 0.08 & 0.7 & 1.1 & 0.04
res33 & 0.01 & -0.02 & 1.8 & -0.3 & -0.44
--------------------------------------------------------------
Significant degradations of cross-data fits.
Hmmm. Not sure what was changed! Uh-oh. I think this was
a switch to the Zhao stepwise regression.
Try 2 modes, but use ampvec:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.09 & 0.07 & 0.9 & 1.3 & 0.05
--------------------------------------------------------------
res11 & 0.13 & 0.08 & 0.4 & 2.8 & -0.11
res12 & 0.09 & 0.07 & 0.6 & 1.5 & 0.03
res13 & 0.11 & 0.07 & 0.6 & 2.0 & 0.01
--------------------------------------------------------------
res21 & 0.00 & 0.00 & 0.0 & -0.0 & 0.01
res22 & 0.00 & 0.00 & 0.0 & -0.0 & 0.03
res23 & 0.00 & 0.00 & 0.0 & -0.0 & 0.02
--------------------------------------------------------------
res31 & 0.08 & 0.07 & 1.3 & 1.0 & 0.05
res32 & 0.08 & 0.07 & 1.0 & 1.0 & 0.06
res33 & 0.06 & 0.05 & 2.2 & 0.3 & -0.12
--------------------------------------------------------------
Not as good as best case for the univariate
fits, but the resx solution is better for GMs.
Try mode-2 only. Maybe the multi-mode fits can't select.
But then mode-1 might spuriously project onto mode-2 basis.
But this ought to be rejected by the goodness-of-fit to
the withheld data.
---> Hmmm. Always getting the same solution. Any time it can
pick up mode-2, it does.
Try 2 modes and make bw of both 0.23 ==> many more mode-2 modes
available to pick from.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.07 & 0.06 & 0.9 & 1.0 & 0.03
--------------------------------------------------------------
res11 & 0.13 & 0.08 & 0.4 & 3.0 & -0.16
res12 & 0.07 & 0.05 & 0.5 & 1.0 & 0.02
res13 & 0.10 & 0.07 & 0.6 & 1.9 & -0.02
--------------------------------------------------------------
res21 & -0.00 & 0.00 & -0.0 & -0.1 & 0.03
res22 & -0.00 & 0.00 & -0.0 & -0.1 & 0.05
res23 & -0.00 & 0.00 & -0.0 & -0.1 & 0.03
--------------------------------------------------------------
res31 & 0.08 & 0.07 & 1.1 & 1.0 & 0.02
res32 & 0.07 & 0.06 & 0.9 & 1.0 & 0.03
res33 & 0.06 & 0.05 & 2.4 & 0.4 & -0.18
--------------------------------------------------------------
---> All-around worse!
Thus, the LARS approach is not really capable of identifying
the significant vs non-significant modes, even with the
withheld data for tuning. ==> The choice for the structure of the
chosen modes is crucial.
Compromise:
2-modes, bw=[0.23 0.11]:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.08 & 0.07 & 1.1 & 1.2 & 0.05
--------------------------------------------------------------
res11 & 0.11 & 0.08 & 0.5 & 2.2 & -0.01
res12 & 0.09 & 0.06 & 0.6 & 1.5 & 0.03
res13 & 0.10 & 0.07 & 0.6 & 1.8 & 0.02
--------------------------------------------------------------
res21 & 0.00 & 0.00 & 0.0 & -0.0 & 0.01
res22 & 0.00 & 0.00 & 0.0 & -0.0 & 0.03
res23 & 0.00 & 0.00 & 0.0 & -0.0 & 0.02
--------------------------------------------------------------
res31 & 0.08 & 0.07 & 1.3 & 1.0 & 0.05
res32 & 0.08 & 0.07 & 1.3 & 1.0 & 0.05
res33 & 0.07 & 0.06 & 1.8 & 0.7 & -0.00
--------------------------------------------------------------
This actually seems to be the best case so far!
As above, and turn off too-small amplitude check (but still omit GFO):
(Interesting, with these small-amp obs included, the permutation
test triggers at step 17, and the OCV optimizations hit the edge.)
==> a little worse than above. Change the test to >7
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.06 & 0.07 & 0.5 & 0.7 & 0.04
resx & 0.06 & 0.08 & 0.6 & 0.8 & 0.04
--------------------------------------------------------------
res11 & 0.06 & 0.09 & 0.6 & 0.9 & 0.04
res12 & 0.06 & 0.08 & 0.6 & 0.8 & 0.04
res13 & 0.06 & 0.08 & 0.6 & 0.8 & 0.04
--------------------------------------------------------------
res21 & 0.00 & 0.00 & 0.0 & 0.0 & 0.01
res22 & 0.00 & 0.00 & 0.0 & 0.0 & 0.03
res23 & 0.00 & 0.00 & 0.0 & 0.0 & 0.02
--------------------------------------------------------------
res31 & 0.04 & 0.07 & 1.3 & 0.5 & 0.05
res32 & 0.05 & 0.07 & 1.0 & 0.5 & 0.06
res33 & 0.03 & 0.05 & 2.2 & 0.1 & -0.12
--------------------------------------------------------------
Interesting: res12 and res32 are individually better wrt GM,
but resx is worse! Not sure how that can be. Aha. optavg2
seems to be using the GM not GMs!?!
Change optavg to use GMs:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.06 & 0.07 & 0.5 & 0.7 & 0.04
resx & 0.05 & 0.07 & 1.0 & 0.6 & 0.06
(all other entries are identical)
Turn the small-amplitude screening back on: (ampcrit=1.5)
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.11 & 0.07 & 0.5 & 1.6 & 0.04
resx & 0.08 & 0.07 & 1.0 & 1.1 & 0.06
Hmmm. It explains more variance wrt HA and HAs, but the relative
change compared to HRET is much worse for HAs.
==> TURN OFF ampcrit SCREENING. (set ampcrit=0.0)
Mid-Indian, based on experiments from Andaman, using 2 modes
with increased bandwidth:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.4 & 0.04
resx & 0.06 & 0.06 & 1.0 & 0.9 & 0.04
Return to the Tuomoto example and see if an outlier screening,
rather than topography limits, is the correct way to do it.
Also, see if GFO is an issue or not. Remember: the GM data were
screened for depth when they were extracted from RADS.
/workdir/n0056/p012/j005/ # Tuomoto SE, post HRET12.0
Indeed: in my new outlier screening plot, based on amplitude, I
can see there are a few spikes. Try to catch them with amplitude-based
screening, and then back off from the topography-based screening.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.68 & 0.81 & 8.0 & 18.2 & 7.31
resx & 0.60 & 0.86 & 21.2 & 17.2 & 8.42
--------------------------------------------------------------
res11 & 0.63 & 0.54 & 0.5 & 27.4 & 3.69
res12 & 0.63 & 0.54 & 0.5 & 27.4 & 3.69
res13 & 0.37 & 0.37 & 3.3 & 16.2 & 3.13
--------------------------------------------------------------
res21 & 0.40 & 0.56 & 6.7 & 11.2 & 6.50
res22 & 0.40 & 0.56 & 6.8 & 11.2 & 6.53
res23 & 0.40 & 0.56 & 6.7 & 11.2 & 6.50
--------------------------------------------------------------
res31 & 0.53 & 0.83 & 22.2 & 14.1 & 8.30
res32 & 0.53 & 0.83 & 22.2 & 14.1 & 8.30
res33 & 0.53 & 0.83 & 22.5 & 14.1 & 8.31
--------------------------------------------------------------
Looks pretty good. And, recall, this is using 2 modes with smaller
bandwidth.
Put GFO back in: (hopefully this will be unequivocally worse or better)
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.59 & 0.81 & 8.0 & 15.2 & 7.31
resx & 0.56 & 0.86 & 20.9 & 15.2 & 8.45
--------------------------------------------------------------
res11 & 0.55 & 0.58 & 1.9 & 22.9 & 4.25
res12 & 0.55 & 0.57 & 1.9 & 22.9 & 4.25
res13 & 0.37 & 0.44 & 3.7 & 15.8 & 3.67
--------------------------------------------------------------
res21 & 0.39 & 0.61 & 7.3 & 10.5 & 7.18
res22 & 0.40 & 0.61 & 7.4 & 10.6 & 7.25
res23 & 0.39 & 0.61 & 7.3 & 10.5 & 7.18
--------------------------------------------------------------
res31 & 0.49 & 0.83 & 22.2 & 12.3 & 8.30
res32 & 0.49 & 0.83 & 22.2 & 12.3 & 8.30
res33 & 0.49 & 0.83 & 22.2 & 12.3 & 8.30
--------------------------------------------------------------
This is very slightly better wrt GMs. Based on optavg2,
it is using slightly more of the HA-based estimate to
get the increased bit of agreement with GMs. Thus, let's keep
the GFO data.
Put in an outlier rejection based on amplitude:
(GM and UV-tuned HA estimates are hitting the upper limit=>
need more modes here!)
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.53 & 0.81 & 8.0 & 13.6 & 7.31
resx & 0.59 & 0.92 & 19.5 & 16.3 & 8.94
--------------------------------------------------------------
res11 & 0.55 & 0.79 & 9.2 & 19.0 & 7.54
res12 & 0.55 & 0.79 & 9.2 & 18.9 & 7.53
res13 & 0.55 & 0.79 & 9.2 & 18.9 & 7.53
--------------------------------------------------------------
res21 & 0.30 & 0.50 & 6.1 & 8.0 & 5.78
res22 & 0.32 & 0.54 & 6.5 & 8.6 & 6.27
res23 & 0.30 & 0.50 & 6.1 & 8.0 & 5.78
--------------------------------------------------------------
res31 & 0.50 & 0.84 & 21.8 & 11.5 & 8.28
res32 & 0.49 & 0.83 & 22.6 & 11.4 & 8.31
res33 & 0.48 & 0.82 & 22.9 & 11.3 & 8.29
--------------------------------------------------------------
Put the small-amplitude cleaner back in, and also use the old stopping
criterion:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.67 & 0.81 & 8.0 & 18.7 & 7.31
resx & 0.74 & 0.93 & 19.5 & 22.6 & 8.95
--------------------------------------------------------------
res11 & 0.72 & 0.82 & 9.9 & 25.6 & 7.64
res12 & 0.71 & 0.82 & 9.9 & 25.5 & 7.60
res13 & 0.71 & 0.82 & 9.9 & 25.5 & 7.60
--------------------------------------------------------------
res21 & 0.45 & 0.62 & 7.4 & 13.7 & 7.31
res22 & 0.45 & 0.62 & 7.4 & 13.7 & 7.33
res23 & 0.45 & 0.62 & 7.4 & 13.7 & 7.31
--------------------------------------------------------------
res31 & 0.63 & 0.84 & 22.1 & 16.6 & 8.30
res32 & 0.62 & 0.83 & 22.6 & 16.5 & 8.31
res33 & 0.61 & 0.82 & 22.9 & 16.4 & 8.29
--------------------------------------------------------------
Wow: everything is improved. It looks like there is still
more signal, though, based on OCV tunings.
Use patchone to look at the fields. Looks reasonable.
Try using max phase speed instead of mean:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.67 & 0.81 & 8.0 & 18.7 & 7.31
resx & 0.69 & 0.89 & 17.4 & 18.8 & 7.42
--------------------------------------------------------------
res11 & 0.59 & 0.69 & 6.7 & 21.5 & 5.93
res12 & 0.59 & 0.69 & 6.6 & 21.4 & 5.87
res13 & 0.59 & 0.69 & 6.6 & 21.4 & 5.87
--------------------------------------------------------------
res21 & 0.30 & 0.43 & 4.3 & 7.9 & 4.54
res22 & 0.32 & 0.46 & 4.6 & 8.4 & 4.79
res23 & 0.30 & 0.43 & 4.3 & 7.9 & 4.54
--------------------------------------------------------------
res31 & 0.60 & 0.81 & 19.8 & 13.9 & 6.99
res32 & 0.60 & 0.81 & 19.8 & 13.9 & 6.99
res33 & 0.60 & 0.81 & 19.9 & 13.9 & 7.01
--------------------------------------------------------------
NOPE. Maybe use mean, but just from within the data window?
Implemented the "extra" value, in degrees, to trim the window
within which cebar is computed. Hmmm. Not sure it will work as
intended.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.66 & 0.81 & 8.0 & 18.7 & 7.31
resx & 0.74 & 0.94 & 19.7 & 22.6 & 9.42
--------------------------------------------------------------
res11 & 0.68 & 0.80 & 9.3 & 25.6 & 8.08
res12 & 0.68 & 0.80 & 9.3 & 25.4 & 8.00
res13 & 0.68 & 0.80 & 9.3 & 25.4 & 8.00
--------------------------------------------------------------
res21 & 0.44 & 0.63 & 7.5 & 13.8 & 8.19
res22 & 0.47 & 0.67 & 7.9 & 14.5 & 8.72
res23 & 0.44 & 0.63 & 7.5 & 13.8 & 8.19
--------------------------------------------------------------
res31 & 0.67 & 0.88 & 22.9 & 16.9 & 8.78
res32 & 0.67 & 0.88 & 22.9 & 16.9 & 8.78
res33 & 0.67 & 0.88 & 23.0 & 16.9 & 8.78
--------------------------------------------------------------
Wow. This is even better than the previous best.
Try it at Andaman's also.
Look at the Andamans: workdir/n0087/p011/j007/
Might consider 500m, rather than 1000m cutoff.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 1.91 & 2.88 & 19.4 & 36.8 & 12.33
--------------------------------------------------------------
res11 & 1.94 & 2.82 & 2.3 & 40.2 & 11.60
res12 & 1.94 & 2.82 & 2.3 & 40.1 & 11.58
res13 & 1.42 & 2.08 & 6.8 & 29.5 & 7.83
--------------------------------------------------------------
res21 & 0.38 & 0.60 & 3.3 & 9.3 & 7.21
res22 & 0.39 & 0.63 & 3.4 & 9.6 & 7.53
res23 & 0.38 & 0.60 & 3.3 & 9.3 & 7.21
--------------------------------------------------------------
res31 & 1.46 & 2.45 & 33.1 & 23.3 & 10.09
res32 & 1.46 & 2.45 & 33.1 & 23.3 & 10.09
res33 & 1.49 & 2.48 & 33.6 & 23.8 & 10.21
--------------------------------------------------------------
Based on the map, I bet my wavenumber estimate is pretty bad.
Drop GFO:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.46 & 3.46 & 13.9 & 40.5 & 13.29
resx & 2.04 & 2.73 & 21.1 & 39.6 & 12.14
--------------------------------------------------------------
res11 & 2.07 & 2.59 & 1.7 & 44.4 & 11.64
res12 & 2.02 & 2.52 & 2.7 & 43.4 & 11.27
res13 & 1.45 & 1.77 & 5.4 & 31.4 & 7.41
--------------------------------------------------------------
res21 & 0.56 & 0.75 & 3.8 & 13.2 & 8.72
res22 & 0.59 & 0.80 & 4.0 & 13.9 & 9.14
res23 & 0.56 & 0.75 & 3.8 & 13.2 & 8.72
--------------------------------------------------------------
res31 & 1.64 & 2.50 & 33.8 & 25.9 & 10.26
res32 & 1.64 & 2.50 & 33.8 & 25.9 & 10.26
res33 & 1.64 & 2.50 & 33.8 & 25.9 & 10.26
--------------------------------------------------------------
==> Keep GFO
Use maximum instead of mean phase speed to compute wavenumber.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 2.00 & 3.19 & 19.6 & 39.2 & 12.33
--------------------------------------------------------------
res11 & 1.93 & 3.01 & 3.1 & 41.3 & 11.85
res12 & 1.90 & 2.96 & 3.5 & 40.8 & 11.67
res13 & 1.47 & 2.31 & 4.9 & 33.6 & 9.05
--------------------------------------------------------------
res21 & 0.77 & 1.25 & 5.3 & 15.5 & 10.51
res22 & 0.80 & 1.31 & 5.3 & 15.9 & 10.82
res23 & 0.77 & 1.25 & 5.3 & 15.5 & 10.51
--------------------------------------------------------------
res31 & 1.60 & 2.78 & 33.3 & 27.5 & 9.97
res32 & 1.60 & 2.79 & 33.7 & 27.5 & 9.99
res33 & 1.60 & 2.80 & 34.0 & 27.4 & 10.02
--------------------------------------------------------------
Interesting: all the stats for resx are notably improved, as are the
res2 fields, for some reason.
Check this approach with Tuomoto SE also.
Try using cebar computed only from near middle of data window.
Compute it properly using the s2 field.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 1.92 & 2.90 & 20.0 & 37.0 & 12.24
--------------------------------------------------------------
res11 & 1.96 & 2.84 & 2.4 & 40.5 & 11.52
res12 & 1.94 & 2.82 & 2.5 & 40.2 & 11.43
res13 & 1.41 & 2.05 & 6.3 & 29.0 & 7.65
--------------------------------------------------------------
res21 & 0.42 & 0.68 & 3.3 & 10.3 & 8.08
res22 & 0.42 & 0.68 & 3.3 & 10.4 & 8.14
res23 & 0.42 & 0.68 & 3.3 & 10.3 & 8.08
--------------------------------------------------------------
res31 & 1.48 & 2.48 & 33.2 & 24.4 & 10.13
res32 & 1.48 & 2.48 & 33.2 & 24.4 & 10.13
res33 & 1.48 & 2.49 & 33.3 & 24.5 & 10.15
--------------------------------------------------------------
This is a slight degradation, but it certainly points out
the importance of having the "correct" basis function.
Use the cemax just from the center of the domain:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 2.07 & 3.14 & 22.2 & 40.3 & 12.94
--------------------------------------------------------------
res11 & 2.04 & 2.92 & 4.6 & 42.4 & 12.23
res12 & 2.04 & 2.91 & 4.6 & 42.3 & 12.18
res13 & 1.46 & 2.16 & 6.1 & 30.2 & 7.70
--------------------------------------------------------------
res21 & 0.55 & 0.87 & 3.9 & 12.9 & 9.04
res22 & 0.55 & 0.88 & 3.9 & 12.9 & 9.07
res23 & 0.55 & 0.87 & 3.9 & 12.9 & 9.04
--------------------------------------------------------------
res31 & 1.64 & 2.80 & 35.8 & 29.6 & 10.92
res32 & 1.64 & 2.80 & 35.8 & 29.6 & 10.92
res33 & 1.64 & 2.80 & 35.8 & 29.6 & 10.92
--------------------------------------------------------------
Is there improvement with a single mode?
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 1.99 & 3.10 & 17.5 & 38.9 & 11.32
--------------------------------------------------------------
res11 & 1.93 & 2.90 & 2.3 & 40.8 & 11.01
res12 & 1.90 & 2.84 & 2.6 & 40.2 & 10.73
res13 & 1.11 & 1.72 & 5.8 & 22.7 & 5.74
--------------------------------------------------------------
res21 & 0.82 & 1.34 & 4.1 & 15.9 & 10.03
res22 & 0.84 & 1.36 & 4.0 & 16.0 & 10.15
res23 & 0.82 & 1.33 & 4.1 & 15.8 & 9.99
--------------------------------------------------------------
res31 & 1.58 & 2.75 & 29.8 & 28.2 & 9.22
res32 & 1.57 & 2.76 & 30.5 & 28.1 & 9.24
res33 & 1.57 & 2.76 & 30.8 & 27.9 & 9.24
--------------------------------------------------------------
Nope. A bit worse.
2 modes, double the bandwidth:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 2.20 & 3.29 & 21.4 & 42.4 & 13.67
--------------------------------------------------------------
res11 & 2.19 & 3.15 & 7.7 & 43.9 & 13.22
res12 & 2.19 & 3.15 & 7.7 & 43.9 & 13.20
res13 & 2.01 & 2.88 & 8.4 & 40.6 & 11.60
--------------------------------------------------------------
res21 & 0.71 & 1.16 & 6.4 & 15.7 & 11.75
res22 & 0.71 & 1.16 & 6.4 & 15.7 & 11.76
res23 & 0.71 & 1.16 & 6.4 & 15.7 & 11.75
--------------------------------------------------------------
res31 & 1.63 & 2.82 & 37.9 & 29.9 & 10.80
res32 & 1.63 & 2.82 & 37.9 & 29.9 & 10.80
res33 & 1.63 & 2.82 & 38.0 & 29.9 & 10.80
--------------------------------------------------------------
Wow! That was unexpected!
Try that with mid-Indian!
===> No improvement. Indeed, given my previous extensive testing of
the Mid-Indian case, it isn't surprising.
Note the sensitivity to the cemax vs cebar, etc, above.
Revisit the issue of estimating the wavenumber from the
PSD of the HA data. Perhaps this is the significant feature which
distinguished HRET from the present approach.
Re-run Mid-Indian using best available 2-mode settings so far
narrow bandwidth, cebar for kwoa, all HA cleanup.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.4 & 0.04
resx & 0.07 & 0.07 & 1.1 & 1.0 & 0.06
--------------------------------------------------------------
res11 & 0.11 & 0.08 & 0.4 & 2.3 & -0.06
res12 & 0.07 & 0.06 & 0.6 & 1.2 & 0.04
res13 & 0.08 & 0.07 & 0.6 & 1.5 & 0.03
--------------------------------------------------------------
res21 & 0.00 & 0.00 & 0.0 & -0.0 & 0.01
res22 & 0.00 & 0.00 & 0.0 & -0.0 & 0.03
res23 & 0.00 & 0.00 & 0.0 & -0.0 & 0.02
--------------------------------------------------------------
res31 & 0.06 & 0.07 & 1.3 & 0.8 & 0.05
res32 & 0.06 & 0.07 & 1.3 & 0.8 & 0.05
res33 & 0.04 & 0.05 & 2.2 & 0.2 & -0.12
--------------------------------------------------------------
Now use kha:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.09 & 0.07 & 0.5 & 1.4 & 0.04
resx & 0.06 & 0.06 & 0.9 & 1.0 & 0.04
--------------------------------------------------------------
res11 & 0.11 & 0.08 & 0.4 & 2.3 & -0.08
res12 & 0.07 & 0.06 & 0.6 & 1.2 & 0.03
res13 & 0.08 & 0.06 & 0.6 & 1.4 & 0.03
--------------------------------------------------------------
res21 & 0.00 & 0.00 & 0.0 & -0.0 & 0.01
res22 & 0.00 & 0.00 & 0.0 & -0.0 & 0.03
res23 & 0.00 & 0.00 & 0.0 & -0.0 & 0.02
--------------------------------------------------------------
res31 & 0.06 & 0.06 & 1.0 & 0.8 & 0.04
res32 & 0.06 & 0.06 & 1.0 & 0.8 & 0.04
res33 & 0.04 & 0.03 & 2.3 & -0.0 & -0.17
--------------------------------------------------------------
There is a slight degradation. -0.02
Now try Tuomoto SE: use kwoa:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.67 & 0.81 & 8.0 & 18.7 & 7.31
resx & 0.75 & 0.94 & 19.3 & 23.1 & 9.41
--------------------------------------------------------------
res11 & 0.70 & 0.82 & 9.4 & 26.0 & 8.21
res12 & 0.69 & 0.81 & 9.4 & 25.9 & 8.16
res13 & 0.69 & 0.81 & 9.4 & 25.9 & 8.16
--------------------------------------------------------------
res21 & 0.41 & 0.56 & 6.7 & 13.0 & 7.19
res22 & 0.41 & 0.57 & 6.8 & 13.1 & 7.26
res23 & 0.41 & 0.56 & 6.7 & 13.0 & 7.19
--------------------------------------------------------------
res31 & 0.67 & 0.89 & 22.7 & 17.2 & 8.73
res32 & 0.67 & 0.89 & 22.7 & 17.2 & 8.73
res33 & 0.67 & 0.89 & 23.0 & 17.2 & 8.79
--------------------------------------------------------------
Using kha:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.67 & 0.81 & 8.0 & 18.7 & 7.31
resx & 0.77 & 0.99 & 19.8 & 23.9 & 9.75
--------------------------------------------------------------
res11 & 0.74 & 0.89 & 10.8 & 27.1 & 8.70
res12 & 0.74 & 0.89 & 10.7 & 27.0 & 8.66
res13 & 0.74 & 0.89 & 10.7 & 27.0 & 8.66
--------------------------------------------------------------
res21 & 0.48 & 0.69 & 8.2 & 14.6 & 8.85
res22 & 0.49 & 0.70 & 8.3 & 14.8 & 8.96
res23 & 0.48 & 0.69 & 8.2 & 14.6 & 8.85
--------------------------------------------------------------
res31 & 0.65 & 0.87 & 22.8 & 15.9 & 8.67
res32 & 0.65 & 0.88 & 23.6 & 15.8 & 8.83
res33 & 0.65 & 0.88 & 23.7 & 15.8 & 8.84
--------------------------------------------------------------
--> WOW! Indeed. Seems to be better. + 0.3
Andamans, kwoa:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 1.96 & 2.96 & 20.6 & 37.7 & 12.52
--------------------------------------------------------------
res11 & 1.98 & 2.87 & 2.3 & 40.9 & 11.65
res12 & 1.98 & 2.87 & 2.3 & 40.8 & 11.64
res13 & 1.55 & 2.26 & 6.0 & 32.2 & 8.70
--------------------------------------------------------------
res21 & 0.51 & 0.83 & 3.7 & 12.1 & 9.42
res22 & 0.51 & 0.84 & 3.7 & 12.2 & 9.48
res23 & 0.51 & 0.83 & 3.7 & 12.1 & 9.42
--------------------------------------------------------------
res31 & 1.51 & 2.54 & 34.0 & 25.0 & 10.31
res32 & 1.51 & 2.54 & 34.0 & 25.0 & 10.31
res33 & 1.52 & 2.55 & 34.1 & 25.1 & 10.33
--------------------------------------------------------------
Andamans, kha:
--------------------------------------------------------------
HRET & 2.25 & 3.46 & 13.9 & 36.2 & 13.29
resx & 2.01 & 3.08 & 21.2 & 38.5 & 12.63
--------------------------------------------------------------
res11 & 2.00 & 2.88 & 3.0 & 41.2 & 12.04
res12 & 1.98 & 2.86 & 3.4 & 40.8 & 11.88
res13 & 1.32 & 1.95 & 6.4 & 26.9 & 6.64
--------------------------------------------------------------
res21 & 0.53 & 0.84 & 3.8 & 12.5 & 8.94
res22 & 0.54 & 0.87 & 3.9 & 12.8 & 9.18
res23 & 0.53 & 0.84 & 3.8 & 12.5 & 8.94
--------------------------------------------------------------
res31 & 1.64 & 2.80 & 34.7 & 28.0 & 10.73
res32 & 1.64 & 2.80 & 34.7 & 28.0 & 10.73
res33 & 1.64 & 2.81 & 34.9 & 28.2 & 10.75
--------------------------------------------------------------
An improvement! + 0.09
Amazon, kwoa:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.62 & 0.75 & 13.1 & 25.4 & 15.85
resx & 0.61 & 0.75 & 13.5 & 25.9 & 15.49
--------------------------------------------------------------
res11 & 0.61 & 0.74 & 12.7 & 26.0 & 15.48
res12 & 0.61 & 0.74 & 12.7 & 26.0 & 15.47
res13 & 0.61 & 0.74 & 12.7 & 26.0 & 15.47
--------------------------------------------------------------
res21 & 0.52 & 0.69 & 11.3 & 22.4 & 14.90
res22 & 0.52 & 0.70 & 11.3 & 22.4 & 14.92
res23 & 0.52 & 0.69 & 11.3 & 22.4 & 14.90
--------------------------------------------------------------
res31 & 0.45 & 0.60 & 18.9 & 18.8 & 11.99
res32 & 0.44 & 0.60 & 20.0 & 18.7 & 12.07
res33 & 0.41 & 0.56 & 21.6 & 17.8 & 11.89
Amazon, kha:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.62 & 0.75 & 13.1 & 25.4 & 15.85
resx & 0.60 & 0.75 & 13.5 & 25.5 & 15.29
--------------------------------------------------------------
res11 & 0.60 & 0.75 & 12.5 & 25.7 & 15.33
res12 & 0.60 & 0.74 & 12.5 & 25.5 & 15.26
res13 & 0.60 & 0.74 & 12.5 & 25.5 & 15.26
--------------------------------------------------------------
res21 & 0.51 & 0.68 & 10.9 & 21.6 & 14.30
res22 & 0.51 & 0.68 & 10.9 & 21.6 & 14.31
res23 & 0.51 & 0.68 & 10.9 & 21.6 & 14.30
--------------------------------------------------------------
res31 & 0.45 & 0.61 & 19.4 & 19.2 & 12.24
res32 & 0.45 & 0.61 & 20.0 & 19.1 & 12.27
res33 & 0.43 & 0.58 & 21.0 & 18.6 & 12.13
--------------------------------------------------------------
Note the overall degradation! -0.2
Indonesian seas (Indo) domain, with kwoa:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 3.76 & 7.10 & 20.6 & 136.5 & 120.30
resx & 3.85 & 7.15 & 43.2 & 174.7 & 121.12
--------------------------------------------------------------
res11 & 3.89 & 7.22 & 42.1 & 176.0 & 122.21
res12 & 3.85 & 7.15 & 43.2 & 174.7 & 121.12
res13 & 2.91 & 5.44 & 55.4 & 141.3 & 94.24
--------------------------------------------------------------
res21 & 2.60 & 5.66 & 3.7 & 100.5 & 132.34
res22 & 2.64 & 5.74 & 2.6 & 101.8 & 134.38
res23 & 1.56 & 3.51 & 15.7 & 62.1 & 82.23
--------------------------------------------------------------
res31 & 0.14 & 0.16 & 45.3 & 3.5 & 1.19
res32 & 0.14 & 0.16 & 45.3 & 3.5 & 1.19
res33 & -1.06 & -0.66 & 168.5 & -22.9 & -29.38
--------------------------------------------------------------
Indonesian seas (Indo) domain, with kha:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 3.76 & 7.10 & 20.6 & 136.5 & 120.30
resx & 4.06 & 7.41 & 29.1 & 175.9 & 121.72
--------------------------------------------------------------
res11 & 4.11 & 7.49 & 28.1 & 177.3 & 123.03
res12 & 4.06 & 7.41 & 29.1 & 175.9 & 121.72
res13 & 3.09 & 5.55 & 49.2 & 135.7 & 89.19
--------------------------------------------------------------
res21 & 2.90 & 6.10 & 3.6 & 104.7 & 132.74
res22 & 2.91 & 6.12 & 3.2 & 105.1 & 133.35
res23 & 1.83 & 3.84 & 15.0 & 62.7 & 82.26
--------------------------------------------------------------
res31 & 0.18 & 0.22 & 47.8 & 4.5 & 1.66
res32 & 0.15 & 0.18 & 37.0 & 4.4 & 1.75
res33 & -2.05 & -1.68 & 177.5 & -68.7 & -54.15
--------------------------------------------------------------
Improvement! +0.6
MARN using kwoa:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.48 & 0.48 & 15.3 & 11.2 & 3.82
resx & 0.44 & 0.57 & 27.2 & 11.1 & 4.95
--------------------------------------------------------------
res11 & 0.45 & 0.47 & 14.4 & 12.1 & 4.01
res12 & 0.45 & 0.47 & 14.4 & 12.1 & 4.01
res13 & 0.45 & 0.47 & 14.4 & 12.1 & 4.01
--------------------------------------------------------------
res21 & 0.16 & 0.19 & 6.5 & 4.4 & 2.17
res22 & 0.16 & 0.19 & 6.6 & 4.4 & 2.18
res23 & 0.16 & 0.19 & 6.5 & 4.4 & 2.17
--------------------------------------------------------------
res31 & 0.43 & 0.56 & 26.4 & 10.3 & 4.74
res32 & 0.40 & 0.55 & 28.3 & 10.1 & 4.90
res33 & 0.40 & 0.55 & 28.3 & 10.1 & 4.90
--------------------------------------------------------------
MARN using kha:
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.48 & 0.48 & 15.3 & 11.2 & 3.82
resx & 0.42 & 0.56 & 27.5 & 10.8 & 4.88
--------------------------------------------------------------
res11 & 0.43 & 0.44 & 14.0 & 11.7 & 3.79
res12 & 0.42 & 0.44 & 13.9 & 11.7 & 3.77
res13 & 0.42 & 0.44 & 13.9 & 11.7 & 3.77
--------------------------------------------------------------
res21 & 0.15 & 0.17 & 6.0 & 4.1 & 1.87
res22 & 0.16 & 0.18 & 6.3 & 4.3 & 1.98
res23 & 0.15 & 0.17 & 6.0 & 4.1 & 1.87
--------------------------------------------------------------
res31 & 0.42 & 0.55 & 26.6 & 10.3 & 4.71
res32 & 0.40 & 0.55 & 28.1 & 10.2 & 4.86
res33 & 0.40 & 0.55 & 28.2 & 10.2 & 4.86
--------------------------------------------------------------
Some degradation! -0.07 The mode-2 wavenumber changes;
mode-1 is nearly identical!
Interesting: There seems to be more signal still to be
extracted at MARN. What if you change the stopping criterion?
Try using kwoa but doubling Lplane. For HA, the optimal tuned wrt
to both GM and UV is still at the maximum. ==> Assume it isn't
the resolution of modes isn't the problem. Just try 0.1 exceedance
criterion. STILL OCCURS! Try 0.5! Seems like I need a different
criterion! GM selects maximum (228), but UV is considerably less
(172); still quite high!
The UV fit goes for 358 and is optimized for GM at 305 and HA at 225 modes.
wrt: HA GM UV HAs GMs
--------------------------------------------------------------
HRET & 0.48 & 0.48 & 15.3 & 11.2 & 3.82
resx & 0.49 & 0.58 & 26.2 & 12.9 & 5.25
--------------------------------------------------------------
res11 & 0.55 & 0.54 & 14.9 & 14.5 & 4.72
res12 & 0.55 & 0.54 & 14.9 & 14.5 & 4.71
res13 & 0.51 & 0.52 & 15.4 & 13.8 & 4.56
--------------------------------------------------------------
res21 & 0.45 & 0.58 & 16.8 & 10.7 & 6.34
res22 & 0.45 & 0.58 & 16.7 & 10.8 & 6.37
res23 & 0.44 & 0.57 & 16.9 & 10.7 & 6.20
--------------------------------------------------------------
res31 & 0.43 & 0.56 & 26.4 & 10.3 & 4.74
res32 & 0.36 & 0.51 & 29.5 & 9.7 & 4.93
res33 & 0.18 & 0.34 & 31.6 & 8.2 & 4.88
--------------------------------------------------------------
Wow: This is quite a bit better than with my usual stopping criterion!
The biggest difference is in the res1* fits to GM. Hmmm.
Test the new stopping criteria:
Indo: Looks good.
Amazon: Also good.
Iceland: Neither good nor bad.
Tuomoto: Much better than was listed above!
Tuomoto SE: Much better. Seems I have been stopping too soon!
Andamans: Much better except for UV. Lol. I have been stopping too early!
Check that location in WPac: Yes. It has improved, too. And this was a region where the
HA fit was significantly worse than HRET.