NEWS

DBHC 0.0.5

smooth.probabilities() treats a NaN as a zero and smooths it to a uniform distribution, as it already did for a row of zeros. Estimation returns a row of NaN for a hidden state it gives no weight to, which made probs <= 0 an NA and stopped hmm.clust() with "missing value where TRUE/FALSE needed", discarding the whole run. Smoothing such a state to a uniform distribution also lets the BIC penalise it, so the search moves away from model sizes that produce one. Found while timing the algorithm over repeated subsamples of the mvad data, where it turns up on a minority of seeds

New vignette school-to-work, a worked case study on the mvad data from TraMineR: 712 school-to-work trajectories of 70 monthly observations over an alphabet of six labour market states. It covers reading a partition, reading a cluster model off its heatmaps, and validating the clusters against background variables the algorithm never saw. The algorithm finds four routes through the six states, and the share of individuals with five or more GCSEs at grades A-C ranges from 13 percent in the training route to 65 percent in the school-to-higher-education route
The vignette documents three properties of the search that are easy to be caught out by, each backed by a study over repeated runs. The rule that stops adding clusters when the BIC stops improving is greedy, and on these data the BIC path is not monotone past the stopping point, so the partition returned is a local optimum. The search is randomised and returned between 2 and 8 clusters across 25 seeds, so one run is a sample from it rather than the answer. And the lowest BIC is not on its own a reason to prefer a partition: the best-scoring run of the 25 buys its score with clusters holding 1 and 10 sequences, and the partitions found disagree with each other substantially, at an adjusted Rand index around 0.2

inst/benchmark/scaling.R reports how the algorithm scales with the number of sequences, as a median over eight seeds per size. A redistribution sweep costs about 0.09 milliseconds per sequence and is linear in their number. The total is dominated by the EM fits and is far less predictable: the same 400 sequences of the mvad data take between 5 and 50 seconds depending only on the seed, because the search chooses its own model sizes and fitting an HMM costs roughly the square of its number of hidden states

cluster.bic() passes its eps argument on to count.parameters(), it always used the default threshold before
hmm.clust() passes its smoothing argument on to assign.clusters(), cluster assignment always smoothed with the default before
smooth.probabilities() returns a uniform distribution for a set of probabilities that are all zero, which used to stay zero and is then rejected by build_hmm()
smooth.probabilities() no longer returns negative probabilities. Absolute discounting assumes the smoothing parameter is smaller than the smallest non-zero probability, and the amount discounted is now lowered to respect that. A probability below the parameter used to be discounted to a negative value, which survived because the row still sums to one, and grew more negative every time the model was smoothed again. Estimates for data with sparse emissions change, see below
select.seeds() stops with a message when too few sequences are left to fill a seed, it used to select NA ids
Loops over sequences and cluster models use seq_along(), they could run an iteration on an empty set

Sequences are evaluated against a cluster model all at once in assign.clusters() and select.seeds() instead of one at a time, which makes the algorithm about five times faster on 1000 sequences and leaves all results unchanged
hmm.clust() has an argument max.iter and size.search() has an argument max.states to bound the search for cluster memberships and hidden states
An infeasible HMM raises a warning instead of printing one
The smaller Swiss Household example is fast enough to run on test again
Both Swiss Household examples are wrapped in \donttest instead of \dontrun, they are slow rather than impossible to run
Results for data with sparse emission probabilities change slightly because of the fix to smooth.probabilities() above. On the simulation study of the accompanying paper nothing changes but the BIC, by 0.0001 percent. On the full Swiss Household data the cluster models and a small number of cluster memberships change, the number of clusters and their sizes do not

Replaced Author and Maintainer with Authors@R
Dropped LazyData, the package has no data directory
Requires seqHMM (>= 2.0.0), the version this release is tested against
Heatmaps refer to their data with the .data pronoun, which silences the deprecation warnings from ggplot2