Ox 1.20: new features and fixes

Read this first

• Recompile DLLs

  This is required because the internal structure of arrays has been made more efficient. Even when your DLL does not use arrays, a recompile is required.

• Recompile .oxo files

  Of course, .oxo files which are part of this release need not be recompiled.

  Note that you can see local variables when using the debugger to trace into .oxo files only if they have been compiled with debugging on (whic leads to slower code).

• check use of vecindex()

  vecindex() now returns an empty matrix if no matches are found. It used to return -1 in that case.

• diff0pow()

  has been renamed to diffpow()

• The following packages need to be upgraded:
  • Arfima
  • DPD
  • EmmPack
  • SSFpack

New features

• Created ox/lib directory with AcfFft.ox, DensEst.ox, HPfilter.ox, NormTest.ox, Quantile.ox, Spline3w.ox. See the top of these files for documentation. Created ox/samples/lib directory with sample programs.
• Added a choice of two new random number generators (with periods 2^60 and 2^113). These are choosen using ranseed().
• Now allowing for installation of a replacement random number generator using a DLL.
• Created an interactive mode. This loads ox_init.ox at start up. Interactive mode does not need explicit declaration of variables in code which is entered.
• The internal code buffer size will adjust as required for large programs. Now allowing setting the sizes of the symbol table and run-time stack on the command line.
• Implemented constant folding for strings as an automatic optimization.
• Created debug functionality. This allows for:
  • break: sets break points;
  • go: up to a break point, or a line in a file;
  • step in/over/out: for tracing
  • show: displays variables the within current scope
  • expressions involving any of the variables within the scope (which also may change variables)
• downloadable console version for NT/95 allows DLLs to be called
• renamed command line programs:
  • oxl.exe for Windows NT/95 console
  • oxdos.exe for Windows 3.1 and MS-DOS 6 (or older)
• Better run-time error messages
• Now using the OXPATH environment variable if present instead of before INCLUDE (OXPATH is preferred, INCLUDE will be used if OXPATH is absent).
• More flexible DLL search: along OXPATH search path, and also trying (e.g. for arfima): OXPATH/packages/arfima/arfima
• The text window created by OxRun in GiveWin could not be saved using File/Save As under windows 3.1. This was because the window title corresponded to a long filename. Now the naming has been changed, e.g. when running a file called c:\temp\test.ox, the text window will be test.out by default, and the graphics window test.

Syntax enhancements

• Warning when concatenating non-matching matrices. This warning can be switched off with oxwarning()
• Better support for matrices with zero dimension; can now use <> to start iterative concatenation.
• Improved result of operations involving an empty matrix
• Removed the transpose/character constant ambiguity, as for example in x'z'
• More flexible array indexing such as: as[1:2] and as[<1:2>].
• Added support for multiple assignment, e.g. [a,b]=c; This can be used to implement multiple returns.
• Variables are now allowed in array expression, e.g. a = {"aap", x};
• Change to array usage: it is now easier to pass arrays as arguments and use them in assignments.
• All library functions now follow type conversion rules in arguments. If a matrix is expected, an int/double will be converted to a matrix. If int or double is expected, the [0][0] element if a matrix argument is used.
• Now allowing index on double or int (for referencing, not for assignment), as well as one index on matrix, e.g.: x[0] = y[2];
• power multiplication, division, and relational operators: scalar matrix behaves as double, so e.g. <1,2> * <2> is as <1,2> * 2.0.
• Infinity support.

New functions

cexp(), clog():

complex exponent, logarithm

Database class:

GetVarByIndex, GetVarIndex(), GetVarNameByIndex, GetSample(), Grow(), ForceSample(), GetGroupNames(), SelectByIndex(), DeSelectByIndex(), DeSelectByName()

decqr(), decqrmul():

QR decomposition

denskernel(), denspoisson():

kernel and poisson densities

dfft():

discrete Fourier transform

diffpow()

fractional difference

discretize():

counts date in regularly spaced intervals

dropc(), dropr():

returns a matrix with specified rows/columns dropped.

gammafact():

gamma function, gamma(n+1) equals n!

isdotinf():

checks for infinity

norm():

various types of norm of a matrix or vector

oxfilename():

returns the current source code name

oxwarning()

controls various run-time warnings

PcFiml class:

Support for I(2) cointegration tests.

p-values of I(1) and I(2) cointegration tests.

PcNaiveDgp class:

Support for breaks in DGP

println(), fprintln():

as print(), fprint(), but ensuring that next output is on a new line.

probpoisson():

Poisson cumulative density function

raninvgaussian(), ranlogarithmic(), ranlogistic(), ranlogn(), ranmultinomial(), rannegbin(), ranstable(), ransubsample(), ranuorder()

new random number generators

Simulation class:

Support for plotting results while simulating

sizec(), sizer(), sizerc():

similar to columns(), rows, rows()*columns(), but returning 1 for a scalar integer or double.

vecrindex():

the vecr version of vecindex()

Deleted functions

• removed spikes() function.
• removed diff0pow, created diffpow.
• Removed ranno()

Function reorganization

• Created an Arma package for the arma0(), armavar(), armagen() armavar(), diffpow() and pacf() functions. These requires the arma.h header file, but are otherwise still built into Ox.
• Created a Probability package for the distributions other than normal, F, t and Chi-squared, and for random number generators other than uniform and normal. These require the oxprob.h header file, but are otherwise still built into Ox.
• moved documentation of MaxBFGS() etc. into maximization package.
• Created ox\lib directory. Also did moved nortest.ox here (as NormTest.ox !); PcFiml and derived classes need not be linked with nortest.oxo anymore.

Function enhancements

Database class:

Clearer about missing values: when loading data in database class: default from -9999.99 to .NaN.

Append() can now use string for one column, append same var twice: give warning.

Renew() can now use array for several columns, will append if new variable is created.

renamed GetMaxLag() to GetMaxSelLag(), SetSelSample() to SetSelSample()

decsvd():

now allowed for any matrix

denst():

can now do double arguments.

eigen():

now returns 1xm and mxm if there are only real eigenvalues, 2xm and 2mxm when there are complex ev's.

fft():

created real fft and inverse real fft

invertsym():

added optional logdet argument

MaxBFGS():

couldn't handle a one parameter problem where the 1 parameter is a double. Also couldn't transpose a 0 x c matrix.

allows 0 or matrix for mHess argument (0: use identity matrix, matrix: use that matrix).

now continues with line search while there is improvement. This could lead to somewhat different results for functions with multiple optima.

pacf():

can solve system (apply filter), and apply inverse filter.

PcNaiveDgp class:

is now more efficient when using NORMAL as distribution. This will affect outcomes a little bit, because random numbers are generated in a different order.

periodogram():

one argument function evaluates T*periodogram at Fourier frequencies. ilag <= 0 now also uses T-1.

print():

Added option to have different format for each matrix column "%cf".

Small chance to default matrix format strings, the column separation is now part of the format, and not automatically included anymore.

Changed default printing for: array, class object, function.

quantilec(), quantiler(), sorting functios

now skips over .NaN (was going in infinite loop during sorting).

rank(x):

is relative to norm, so rank(1e-100) is 1; created two argument version.

ranseed()

allows choice of two new random number generators.

solvelu():

can be used to solve triangular systems.

unit():

created two argument version

vecindex():

created two argument version

vecindex()(), deletec() (etc.), and selectifc() (etc.):

return <> if the result is empty.

vecindex(), sortbyc(), sortbyr(), limits(), submat(), strfind():

now take array_base into account.

zeros(ma), ones(ma), and constant(d,ma);:

when ma is matrix: uses dimensions of ma to create the matrix.

Function bug fixes

Database class:

.fmt extension for formatted data file in to confused it.

denst():

was wrong for odd df's (it was using int(df/2)) this was most noticable for t(1) (Cauchy) distribution.

imod(x,0):

crashes out.

isdotfeq():

did not work on two scalar arguments

logdet():

didn't set sign for 1x1 mat argument also didn't take logarithm in that case.

ols2c(), ols2r():

had potential problem when using singular design matrix, XtXinv argument is set to 0 when singular.

pacf():

did not compute partial autocorrelations correctly

ranf():

converted degrees of freedom to an integer

timespan():

better writing of lapsed time (especially for >100 hrs) Time text went wrong when it is long, e.g. > 100 hours.

Bug fixes:

• Did allow calling of non-static members from outside class (e.g. Test::Print() from main).
• Runtime error: 'matrix[5][100] * matrix[1][5]' bad operand should have been / not *
• Testing for NaN was not strictly correct on Unix boxes.
• Error with Ox output via oxrun into GiveWin when printing matrix with column labels.
• Creating .oxo file with non-square matrix constants was a problem.
• OxRun from command line couldn't handle file names with spaces in double quotes.
• In rare cases it was possible that an old matrix from invert() gets reused in rann(). This could then create a problem.
• Statements like ma = ma; had an undefined result.
• There was a potential memory leak in `tabular' expressions.

Known problems in version 1.20 which are solved in 1.20a:

• An optimization bug in the Windows compiler caused every second call to rann(1,1) to return 0 (Windows 95/NT only).
• Printing a matrix with column headers, or array/class didn't work properly when using OxRun (Windows only).
• Sample selection in database derived classes didn't work properly with lags or initial missing values (all versions)
• Infinite loop when reading PcGive files (MS-DOS version only).