WEIAN DATA 惟安数据科技
HOME / IN-HOUSE TOOLS & CASE STUDIES

One toolchain across
"modeling → weighting → merging"

Not scattered scripts: three interlocking R packages built around the full data flow of large-scale assessment surveys. All developed in-house with full IP ownership; no client data involved.

Raw response data
IRTCIRT modeling ratecalibCalibration weighting mergecalibUnit merging
Trustworthy conclusions
CASE 01
IRTC
High-performance IRT estimation engine
Item Response Theory C++ / Rcpp Streaming low-memory 26 test suites CRAN upcoming
■ THE PROBLEM

Large assessments involve hundreds of thousands to millions of test-takers, many items, and multiple dimensions. Traditional IRT software fails at this scale — memory explodes or estimation never finishes — and commercial packages are hard to customize. Research teams need an engine that is both rigorous and able to run big data on an ordinary workstation.

■ THE SOLUTION

IRTC is a self-contained marginal maximum likelihood (MML) estimation engine with C++ (Rcpp / RcppArmadillo) internals, systematically engineered for scale:

  • Model coverage: Rasch/1PL, PCM, RSM, 2PL, GPCM; unidimensional and between-item multidimensional; latent regression, multi-group analysis, case weights, and EAP ability estimation.
  • Performance: multithreaded parallel E-step, block-wise streaming posterior computation (no full person × node posterior matrix), dimensional decomposition, analytic gradients with hybrid Newton M-step, automatic engine routing.
  • Trustworthiness: optional node pruning is accompanied by stratified empirical error checks with a written report — the exact path is the default, and statistical targets are never silently changed.
■ RESULTS
  • The streaming engine has completed million-sample, multidimensional GPCM estimation in prototype benchmarks, compressing memory needs to ordinary-workstation levels.
  • Three engines (auto / grid / streaming) with automatic routing that records its reasoning — usable by default, controllable when needed.
  • 26 automated test suites and a beginner-friendly manual; engineering quality on par with established open-source projects.
What it means for your project: the flagship proof of WEIAN's "IRT expertise + large-scale engineering" claim. It serves as the core engine for custom assessment modeling projects and can be tailored and extended to your requirements. New to IRT itself? See Methods · CTT vs. IRT.
CASE 02
ratecalib
Multi-subgroup target-rate calibration weighting
Calibration weighting Convex QP 13 test suites MIT Published on CRAN
View on CRAN →
■ THE PROBLEM

Surveys often need overall and multiple overlapping subgroup pass rates (by region, gender, urban/rural) to approach or exactly match given targets — without breaking the original sampling weight structure or population margins. Manual reweighting is neither rigorous nor reproducible.

■ THE SOLUTION
  • Formalizes the requirement as bounded convex quadratic programming (QP), solved at the population-unit × outcome aggregation level — rigorous and efficient on large samples.
  • Two modes — soft (approach targets) and exact (match exactly) — both preserving the initial weight structure and overall margins. Methodologically grounded in Deville & Särndal (1992) calibration theory.
  • Practical tooling: one-step interface, pre-solve data checks, target-table builders, effective sample size (ESS) and design-effect diagnostics, example data; powered by the high-performance osqp solver.
■ RESULTS
  • Published on CRAN under the MIT license, with 13 automated test suites.
  • Turns a real-world rate-calibration challenge into a reproducible, diagnosable standard workflow.
What it means for your project: it demonstrates WEIAN's ability to translate a business requirement into a mathematical optimization problem and ship it as a tool — a need that recurs across survey projects of every kind. For the underlying theory, see Methods · Calibration weighting.
CASE 03
mergecalib
Deterministic unit merging under interval calibration targets
Integer programming (MILP) Survey post-processing 16 test suites MIT CRAN upcoming
■ THE PROBLEM

Tabulating graded results by population units (province × gender × urban/rural × age × education) often leaves cells with tiny or zero samples, destabilizing proportion estimates. Units must be merged so that provincial and national subgroup grade shares (e.g., A/B/C/D) stay within target intervals and every final unit has positive samples — a constraint-heavy combinatorial problem that is extremely error-prone by hand.

■ THE SOLUTION
  • Solves within-province merging as set-partitioning mixed-integer linear programming (MILP).
  • Lexicographic objectives: minimize merged sample size, then population distance, outcome heterogeneity, weight distortion, and merge count — in that order.
  • Honest by design: when constraints are infeasible, it reports so explicitly instead of silently compromising the targets.
■ RESULTS
  • Deterministic, reproducible merging plans replace cell-by-cell judgment calls.
  • Open-sourced under MIT with 16 automated test suites; CRAN submission upcoming.
What it means for your project: together with ratecalib it covers the post-processing stage of survey data — the unglamorous step where many published estimates quietly go wrong. For the underlying idea, see Methods · Sparse unit merging.
* All three packages are developed in-house by WEIAN with full IP ownership. Case studies contain no client project names or sensitive data.

Want tools like these,
built for your project?

Email us →
Custom development, tailoring, and extension of the toolchain are all available — initial assessment is free.
See how we work.

contact@weiandata.com