2002-02-16 Carlos Jimenez * qpack-1-0-3 * New fields for retrieving calibration missing in qpack. 2002-02-13 Patrick Eriksson * qpack-1-0-2 * Included two different ways of retrieving calibration variables. The two ways are denoted as TB_REFLOADS and PROPCAL. See README for descriptions. The calibration options handled by the plotting functions. * Main/qp_x2y.m: Added this function to handle the new calibration retrieval. The purpose of this function is to handle all parameters that are not included in ARTS.The baseline fit is moved to this function from qp_x2arts. 2002-01-22 Patrick Eriksson * qpack-1-0-1 * Finished the implementation of continuum absorption that works with an updated version of ARTS. No heavy testing but it seems to work. It is important not to include too high altitudes as the condition number for a matrix to invert as part of OEM becomes too poor. * Made the function qpcls_errorplot that cpmplements qpcls_invchar by plotting the error for individual sources. * The lower left figure created by qpcls_invchar shows now the correlation "averaging kernels" instead of the correlation FWHM. 2002-01-18 Oliver Lemke * qpack-1-0-0 * Version released with arts-1.0. 2002-01-18 Oliver Lemke * qpack-1-0-0-rc1 * OK, this is release candidate 1. Just tagged this as a final version for testing. Hope next version will be the final release. 2002-01-18 Patrick Eriksson * qpack-1-0-0-pre1 * Main/qp_invchar.m: Included functionality for continuum absorption and baseline parameters. * Continuum absorption: Changed the definition for CONTABS_LIMITS so it is easier to adjust to F_MONO. See README. The internal unit for continuum absorption is now 1/km. This to get x-values closer to 1 and get a better rank when inverting matrices for OEM. The unit 1/km is just used internally, retrieves values are in 1/m. Adapted qp_Sxb and qpi_K to these changes. HOWEVER, the continuum absorption is not totally working and cannot be used. There will be an error if you try. Continuum absorption has to wait until 1.1. * Main/qp_contabs_freqs.m: Added this function to handle frequencies for continuum absorption. * Main/qp_abs.m: Added this function to calculate the absorption for a Q (corresponding to qp_y). 2002-01-16 Patrick Eriksson * qpack-1-0-0 (Tag removed) * README: Some small changes and some new text. * Main/qpack.m: Added some to the header text. * CLS/qpcls.m: Wrote some xtext for the function header. 2002-01-15 Patrick Eriksson * qpack-0-0-49 * Setup/qpopt_fmono_by_interp.m: Renamed to qpopt_fmono. See below. * Setup/qpopt_fmono.m: The function qpopt_fmono_by_interp is renamed to this name. This as the old name did a wrong impression and I don't think there will be another similar function, so this name should suffice. A small bug fixed. Heading text improved. * Setup/qpopt_linefile.m: Created and added this function that generates a linefile for a frequency band. The strategy for the function resembles the one for qpopt_fmono. See the function header for more details. 2002-01-12 Patrick Eriksson * qpack-0-0-48 * README: Added description of the fields ANTENNA_MOVE and MEASNOISE_SCFAC. These fields allow that the integration time differs between the zenith angles. * Main/qp_H.m: Included Q.ANTENNA_MOVE. * Main/qp_Se.m: Included Q.MEASNOISE_SCFAC. See README. 2002-01-04 Patrick Eriksson * qpack-0-0-47 * qpopt_fmono_by_interp: Small changes to the messages given. * Some fixes in the sample case. 2001-12-10 Patrick Eriksson * qpack-0-0-46 * qpcls.m: Small changes to the godness structure G. The field names are now all in lower case. 2001-12-10 Patrick Eriksson * qpack-0-0-45 * The Q-fields F_SENSOR and ZA_SENSOR are replaced with the fields BACKEND_FREQS and ANTENNA_ZA, respectively. These names are much more informative than the old ones, they tell you the real meaning, and there is no risk of confusion with the f_sensor and za_sensor variables created by qp_H. Sorry, this means that you have to change your control files. I have been thinking about this change a long time and I tried to avoid it. However, yeasterday I myself got confused and lost quite some time, and I decided to make this change. * CLS/qpcls.m: The cost is now normalised with the length of y to make it more easy to check if the final cost is OK. If you have described the observation uncertainties correctly, the final cost should be close to 1, otherwise you haven't found the minima of the cost function. I cannot say what values above 1 of the final cost that can be accepted, but it should probably not be much above 1. 2001-11-29 Patrick Eriksson * qpack-0-0-44 * Another try to remove the OEM folder. 2001-11-28 Patrick Eriksson * qpack-0-0-43 * CLS/qpcls.m: Some cleaning in the function to make the code nicer. I did some actual changes, without changing the basic algorithms, so there could be a warning for bugs. * Main/qpack_version.m: Added this function to enable and handle version nmumber for Qpack. NOTE: This file should be edited before doing a commit. * Main/qpack.m: The command 'qpack -v' gives the version number. This feature was added to be consistent with ARTS. 2001-11-28 Patrick Eriksson * qpack-0-0-42 * Fixed a bug for qpcls appearing for linear retrievals and when fitted spectra were calculated (it worked if the non-linear gamma fields were specified). * Updfated the Sample folder to follow the changes in ARTS for continua. 2001-11-22 Viju O.-J. * qpack-0-0-41 * Templates/iter1.tmplt: Removed the keyword "CloudSat" from AtmFromRaw. * Templates/basic.tmplt: Removed the keyword "CloudSat" from AtmFromRaw. 2001-11-22 Patrick Eriksson * qpack-0-0-40 * Main/qp_eground_lims.m: Put in warning message if a ground emission range is totally outside F_MONO. * Main/qp_x2arts.m: Fixed a bug for cases when part of F_MONO is outside the ground emission ranges. E_GROUND is now read for each iteration. 2001-11-21 Patrick Eriksson * qpack-0-0-40 * qpack_startup.m: The Setup folder added to the search path. * Main/qp_ympb.m: A function similar to qp_y, but this one returns the monochromatic pencil beam spectra (MPB), that is, H is not applied. The function allows also that F_MONO and ZA_PENCIL are changed from the definition in Q. * Setup/qpopt_fmono_by_interp.m: The first function in Setup. This function "optimizes" f_mono with respect to interpolation error. See the function header for further information. 2001-11-18 Patrick Eriksson * qpack-0-0-39 * CLS/qpcls.m: It is not now demanded that gamma=0 to stop the iterations. The lower treshold for gamma is removed. If the upper limit for gamma is reached, the iteration number is now set to 9999. 2001-11-16 Patrick Eriksson * qpack-0-0-38 * Main/qp_x2arts.m: As suspected, there was a bug for the ground emission, that now is fixed. A non-linear inversion test for AMSU-B worked very well. 2001-11-15 Patrick Eriksson * qpack-0-0-37 * Non-linear CLS inversions did not work. The code is running but something is wrong! What? Maybe the ground emission. 2001-11-15 Patrick Eriksson * qpack-0-0-36 * README: A new Q-field, EGROUND_MINMAX: A vector with length 2 specifying the lower and upper limit for allowed values on the ground emission. If the vector is set to be empty, all values are allowed. These limits are applied only during the iterations of an inversion. When convergence has been reached, the retrieved values are returned even if they are outside the given min and max values. The min/max values can accordingly be used to keep the ground emission inside the range 0 and 1 during the iterations to avoid unphysical conditions. * Main/qp_invchar.m: Plotting for ground emission now included. 2001-11-15 Patrick Eriksson * qpack-0-0-35 * Some extra information in README about definition ot covariance matrices. 2001-10-26 Carmen Verdes * qpack-0-0-34 * I removed my last changes and I returned back to the old version ( qpack-0-0-31 ) 2001-10-26 Patrick Eriksson * qpack-0-0-33 * I just discovered that the OEM directory still existed. This is a new attempt to remove OEM. 2001-10-23 Carmen Verdes * qpack-0-0-32 * fast and dirty changes needed for intercomparison: * basic.tmplt: I added a new flag Q.READ.ABSCOEFF for reading the absorption coefficients. If this is set in the structure then the absorption coefficients are read from the specified file, otherwise they are calculated. This is needed for ARTS intercomparison case 3. Also I changed the template file to output the y_mono , f_mono, za_pencil vectors. * qp_y.m: I changed it to output the y_sensor vector. 2001-10-22 Patrick Eriksson * qpack-0-0-31 * It is now not necessary to always specify F_SENSOR and ZA_SENSOR. These Q-fileds are only needed when you include a backend and an antenna pattern, respectively. The function qp_H now stores the frequency and zenith angle grids that corresponds to your instrument simulation. See further README. * Ground emission is now a retrieval/error variable. See README for instructions on how to specify the input. This part has not been tested in detail. I leave this to Viju. Viju, can you test this? If you find anything strange, please send me your control file (the file where Q is specified) and needed input to run the case. 2001-10-18 Carlos Jimenez * qpack-0-0-30 * Test 2001-10-18 Carlos Jimenez * qpack-0-0-29 * Bug in qp_rnd_atm when doing uniform poiting fixed. * Added CLS/qpcls_batch.m, as qpcls but to invert a set of spectra produced by qp_rnd_atmxsensor.m, see the help function. 2001-10-15 Carlos Jimenez * qpack-0-0-28 * Bug in hse part in qp_rnd_atm fixed * New Q field Q.POINTING_COR, set to 1 the pointing error for all z_tan in a scan are prefectly correlated, set to 0 the errors are uncorrelated. Should we go for a more general approach as with the other errors? * qp_Hd hRedLimb modified, if whole Kx and equal retrieval grid for all species reduction only done for first one as it is identical for all TAGS. 2001-10-15 Carlos Jimenez * qpack-0-0-27 * New Q-field: OEM_GA_UPPER_LIMIT to set an upper limit to gamma, some inversions can go funny -numerical problems?-. qcls.m udpated. * New function qcls_batch.m, as qcls but taking as input a structure used to generate a random set instead of y and doing the inversion of all the spectra in the random set. See the function help. * qp_Hd updated by including hRedKx.m and hRedLimb.m as subfunctions, also updated. * qp_H updated, removing do_reduction check and using defalult values from qpack. * qp_rnd_atm and qp_rnd_sensor updated to include the possibility of give a previous random structure Qr to copy rnd realizations for the new Q instead of generating them for Q (valid e.g if Q and Qr have different sensor but the rnd realizations are needed to be the same) * new Q field: LRED_INDTAGS, for the limb reduction, set to 1 uses for in the reduction only the Kx for the corresponding retrieval tag, set to 0 uses the whole Kx. 2001-10-15 Patrick Eriksson * qpack-0-0-26 * Big internal changes, but just small changes in the control files. Starting with Q-field changes: * New Q-field: RETRIEVAL_METHOD The retrieval method must now be specified. So far is 'oem' the only allowed option, but there will bemore later (see below for CLS). * New Q-field: RECALC_ABS_ON See README. Can be left out. * Big news! The do-levels 1 and 2 are now fully working (before it was only do-level 2 for noise that worked). Accordingly, much more variables can now be placed in Se. For variables beside noise, this is done by adding H*Kb*Sb*Kb'*H to Se (for variables with _DO=2). Note that data reduction can be needed to have enough memory. * The inversion characteristion is now handled in a general function called qp_invchar. For CLS, use qpcls_invchar. In the inversion characterization, variables with _DO = 1 or 2 are included in Se. * As Se is not needed for doing an inversion and in the inversion characterization Se can be different, Se is not longer saved by QP_SE and input/output of QP_SE is changed. If you will do an OEM (CLS) inversion, you shall now use QP_SEINV (qp_Seinv). See further below. * OEM functions and folder renamed to CLS. CLS stands for constrained least squares which will be used as a common name for OEM, Tikhonov regularization and (ordinary, weighted) least squares. So far is only OEM handled, but I plan to include the other two options some day (or night, as right now?) in the future. The functions qpcls and qpcls_invchar are used exactly as qpoem and qpoem_invchar. * Some parts are moved out from qpcls and are called qp_x2arts and qp_x2X. * qp_ycalc renamed to qp_y to be consistent. * Started implementation of ground emission but not yet finished. * To summerize, the standard set of function calls to make an OEM inversion is now: qp_Sx(Q); qp_H(Q); qp_Seinv(Q); qpcls(Q,y); To simylate a measurent: y = qp_y(Qy); 2001-10-10 Carlos Jimenez * qpack-0-0-25 * qp_rnd_atm fixed to be able to get parameterizatrions as retrieval tags e.g. rnd realizations of H2O-PWR98 means disturbing H2O profiles, not H2O_PWR98 profiles! 2001-10-09 Carlos Jimenez * qpack-0-0-24 * Changes in qp_rnd_atm, hse added to the rnd realizations, so if Q.HSE_IN_ON is set to 1 and temperature rnd realizations are done, the correponding z_abs for each t_abs fulfill hse according to the Q.HSE fields given in the Q srtucture defining the rnd data. The hse is calculated by arts/ami/physics/hseCalc, mscript that mimics the hse calculations inside arts, to get consistent results. Q.ALTIUDE_DO added in Templates/rnd.tmplt to flag do_z. * Changes in qp_rnd_atm, Q.POINTING_PDF added so the pointing offset can be defined as 'gaussian' or 'uniform'. As it will be nearly always gaussian, as default is set to 'gaussian', so if Q.POINTING_PDF is not given 'gaussian' is assumed. 2001-10-05 Patrick Eriksson * qpack-0-0-23 * Just included some references to AUG in README 2001-09-28 Patrick Eriksson * qpack-0-0-22 * Mainly changes to sync Qpack with all changes in ARTS. * qpoem can be runed for non-linear inversions again. * Continued work on retrieval of continuum absorption, but not finished. * ycalc.tmplt replaced with basic.tmplt. 2001-09-28 Patrick Eriksson * qpack-0-0-21 * Kx is now an output argument beside Dy when calling qpoem with only one input argumnent. 2001-09-27 Carlos Jimenez * qpack-0-0-20 * cleaning in /Main, qp_rnd, qp_rndxH, not in use any more 2001-09-20 Patrick Eriksson * qpack-0-0-20 * Small modifications in qpoem_invchar. The vertical resolution is not longer a return argument. 2001-09-17 Carlos Jimenez * qpack-0-0-19 * qp_rnd_sensor updated _DO > 0 makes realizations 2001-09-17 Carlos Jimenez * qpack-0-0-18 * qp_H saving also Hs (sensor matrix without reduction) * qp_Hd alowing all + hRedKx or hRedLimb * README updated in NUMBER_DO 2001-09-17 Patrick Eriksson * qpack-0-0-17 * Piecewise polynomial baseline fit included (PPOLYFIT). See README for more details. * The code of qp_Sx changed, but not the functionality (I hope). * The code of qp_Se also changed. The definition of the MEASNOISE_DO field changed to be consistent with the other do fields. Calibration noise can now be used again, but there is now no variable to tune the correlation between different viewing angles. A perfect correlation is assumed. * The field APRIORI_VMR_DIR changed to APRIORI_VMR as the field does not give the name of a directory. I know that this results in that you have to change a lot of control files, but this thing has been bugging for a long time and it is better do the change now than later. * The Sample directory updated to work with the new way of defining the thermal noise. 2001-09-03 Carlos Jimenez * qpack-0-0-16 * Big changes!!! * added qp_rnd_atm, qp_rnd_sensor, qp_rnd_atm_sensor mscripts to generate random sets, see the mscripts help to find out about their usage. The qp_rnd, qp_rndxH and qp_rndcheck are removed. The new mscripts do practically the same, but now the random realizations are done by Matlab mscripts, as the corresponidng part in arts is going to be removed. rnd.tmplt has also been updated to remove the old stuff and generate spectra in brightness temperature, after Qpack change of emission unit. * qp_H and qp_Hd slighly modified to allow the use of /ami/hmatrix/hRedKx and /ami/hmatrix/hRedLimb, the 1st one is reduction based on Kx, the 2nd a special reduction to use with limb observations optimized for each retrieval point and tag (obviously only to be used with a suitable reduction method, e.g an ensemble of neural nets). See the hRedKx and hRedLimb help and README for info. Notice that hRedLimb is still under tests, so do not use till further notice. * qp_Se updated to do rnd realizations using mscript sFromFile Major change, definition of measurement noise, check README. Only measurement noise, calibration to be decided how to implement it, other _DO = 2 also not implemented as in original qp_Se. * qp_Sx updated to also use sFromFile polynomial fit incorporated, continua still not implemented. * README is updated detailing new fields and so on. 2001-08-31 Patrick Eriksson * qpack-0-0-15 * The inclusion of polynomial fit finished. * It is now (more?) important to set f_sensor and za_sensor. These files will be used in some cases even if the corresponding sensor part is not included. If the backend or antenna not are included set F_SENSOR=F_MONO or ZA_SENSOR=ZA_PENCIL, respectively. * A bug in qpoem for non-linear inversion found. Not fixed!! 2001-08-31 Patrick Eriksson * qpack-0-0-14 * Started to include polynomial fit of baseline ripple. Not finished. * The unit used now for emission spectra is Rayleigh-Jeans brightness temperature. One reason to this is that the Sx matrix easily is singular when making baseline fits and using intensities. (I have written in AUG that brightness temperatures cannot be used for practical retrievals, but I think that that statement is wrong if the calibration is done correctly. More about this later. For simulations there is no problems.) 2001-08-24 Patrick Eriksson * qpack-0-0-13 * qpoem_inchar has now return variables. 2001-08-06 Carlos Jimenez * qpack-0-012 * /Main/qp_rnd changed so the spectra is saved without any sensor matrix applied, /Main/qp_rndxH added to do the multiplication. Why? Provides flexibility as it is possible to apply a previous reduction from a structure Qi to a new rnd set given by Q. See help. 2001-07-30 Carlos Jimenez * qpack-0-0-11 * Reduction based on weighting functions implemented, qp_Hd.m updated and new ami function/arts/ami/hmatrix/hRedKx.m added. New fields in Q structure KRED_ON, KRED_N, KRED_DEPTH. * Functions to generate random spectra /Main/qp_rnd and corresponding template /Templates/rnd.tmplt added, based on arts methods to generate random realizations, only species, temperature and pointing. They are PROVISIONAL, as a more detailed implementation not relying in arts methods is needed. 2001-06-13 Patrick Eriksson * qpack-0-0-10 * Smaller changes to cfile templates and qpoem. 2001-04-24 Patrick Eriksson * Qpack-0-0-9 * Put in temperature retrieval with HSE. Two new Q fields: PREF and ZREF. * Updated templates to changes in ARTS regarding HSE, the ground and refraction. * Fixed a bug in qpoem_invchar. 2001-04-18 Patrick Eriksson * Qpack-0-0-8 * More work on qpoem. Bugs fixed and some changes for input and output. Test retrievals have been performed for species, pointing off-set and temperature (yet only without hydrostatic eq.). However, the sample case is not suited for temperature retrievals but the iteration seems to work properly. * A maximum number of iterations introduced (OEM_MAX_ITER). * The positive constraint for non-linear species retrievals can now be toggled (SPECIES_POSITIVE_ON). No heavy tests for this option, but so far it looks good. 2001-04-17 Patrick Eriksson * Qpack-0-0-7 * Added functions etc. to easily simulate measurements corresponding to a Q structure. Noise is not yet included. The function doing the work is called qp_ycalc. A suitable Q for this function si found in Sample and it is called ycalc_q. * The Q-fields for the apriori data are renamed. See README. This to make it possible to select VMRs and PTZ from different directories. * Continua added. 2001-04-12 Patrick Eriksson * Qpack-0-0-6 * The OEM retrievals seem to work for species and pointing off-set. No bigger bug checks. 2001-04-11 Patrick Eriksson * Qpack-0-0-5 * Mainly a back-up commit. * I have started to work on the retrieval stuff (qpoem.m etc.) 2001-03-29 Patrick Eriksson * Qpack-0-0-4 * Included: oem_iter1.tmplt, qp_Se.m, qp_kinfo.m, qpi_Kx.m, qp_Hd.m, qpoem_invchar.m and qpoem_iter1.m * Modefied the sample files somewhat to better match Odin. * Changes to README and sample_q 2001-03-26 Patrick Eriksson * Qpack-0-0-3 * Made an organization into sub-folders. * The README file extended and a first version almost ready. * Created a sample case (found in Sample/). 2001-03-21 Patrick Eriksson * Qpack-0-0-2 * First, not finished, version of the README file. 2001-03-19 Carlos Jimenez * Qpack-0-0-1 * Initializing Qpack under cvs