Sequential testing, CUPED variance reduction, multi-armed bandits, and AI-powered experiment design — all completely free.
Java, Python, React & JS SDKs · SOC 2 compliant · Deploy to your AWS
Free Preview · All features included · Self-hosted on your AWS
From feature flags to enterprise-grade statistical methods — all included
Core Platform
Run multivariate experiments with statistical rigor. Built-in significance testing, confidence intervals, and automated SHIP / KEEP / CONTINUE recommendations.
Deploy safely with gradual rollouts, staged schedules, and kill switches. Bulk toggle, SSE audit stream, and full change history included.
See results as they happen. Trend charts, cumulative and daily views, sample size meters, and days-to-significance estimates.
Automated guardrails monitor error rates and performance. Auto-rollback on anomaly detection protects your users from bad releases.
20+ targeting operators including semantic versioning, geo-distance, time windows, and JSON path. Rules engine evaluates 125K+ ops/sec.
SDKs for Python, JavaScript, Java, and React. RESTful APIs with OpenAPI docs. RBAC with custom roles — ADMIN, DEVELOPER, ANALYST, VIEWER.
Advanced Statistical Methods
mSPRT-based early stopping with always-valid confidence intervals. Stop experiments early when you have evidence, without inflating false positive rates. Alpha spending via O'Brien-Fleming and Pocock boundaries.
Reach significance 20–40% faster using pre-experiment covariate adjustment (OLS θ). Winsorization handles outliers. Works for both conversion and numeric metrics.
Maximize conversions during experiments. Choose Thompson Sampling, UCB1, or Epsilon-Greedy. Background scheduler auto-reallocates traffic to winning variants.
Detect cross-experiment interference before it corrupts your results. Jaccard overlap analysis, chi-squared interaction tests, novelty effect detection, and SUTVA violation alerts.
Claude API–powered hypothesis generation and design suggestions. MCP Server integrates with Claude Code, Cursor, and other coding assistants so your AI tools can design and interpret experiments.
Query experiment results directly in Snowflake, BigQuery, or Redshift. SQL-sanitized sync keeps your warehouse as the source of truth.
Traffic Management & Governance
Prevent experiment contamination using consistent-hashing traffic partitioning. Global holdout groups measure the cumulative impact of your entire experimentation program.
Break down results by device, country, plan, or any user attribute. Bonferroni-corrected per-segment analysis with heterogeneous treatment effect (HTE) detection.
HMAC-SHA256 signed tamper-proof audit trail with real-time SSE stream. SOC 2 Type II & ISO 27001 compliance reports with CSV/JSON export. Custom RBAC roles and effective permissions resolution.
SDK Ecosystem & Integrations
Native Java integration with Spring Boot auto-configuration starter. OkHttp client, LRU+TTL caching, and MD5-based consistent hash bucketing for stable variant assignment.
React hooks and HOC — useFeatureFlag, useExperiment, useMultipleFlags, and withExperimentation HOC. SSR support via ServerClient with no hydration mismatch.
Jira, Salesforce, and GitHub webhooks — sync experiment status, automatically create issues, and push results to your existing toolchain without leaving the platform.
Beta-Binomial posteriors, Monte Carlo PtBB simulations, Bayes Factor (BF10) via Savage-Dickey, credible intervals, and an automatic Bayesian stopping rule — alongside your frequentist analysis.
Server-side URL splitting via Lambda@Edge — no client-side JavaScript, works with any frontend framework. Persistent cookie-based assignment ensures a consistent experience across visits.
5-step guided wizard for experiment creation — define hypotheses, set variants, choose metrics, configure targeting, and launch — all without writing a line of code.
Seamlessly integrate with the tools you already use
Get started in minutes — SDKs for Python, JavaScript, Java, and React
import { Experimently } from '@experimently/sdk';
const client = new Experimently('YOUR_API_KEY');
// Get variant + track conversion
const variant = await client.getVariant(
'checkout-redesign', userId
);
client.track('purchase_completed', userId, {
revenue: 99.99
});
// Feature flag with targeting
const isEnabled = await client.isEnabled(
'new-checkout', userId,
{ country: 'US', plan: 'enterprise' }
);// application.properties
experimently.api-key=YOUR_API_KEY
experimently.cache.ttl-seconds=60
// Auto-configured via Spring starter
@Autowired
ExperimentlyClient client;
String variant = client.getVariant(
"checkout-redesign", userId
);
boolean enabled = client.isEnabled(
"new-checkout", userId,
Map.of("country", "US", "plan", "enterprise")
);import {
ExperimentlyProvider,
useFeatureFlag,
useVariant,
} from '@experimently/react';
// Wrap your app
<ExperimentlyProvider apiKey="YOUR_API_KEY">
<App />
</ExperimentlyProvider>
// Use in any component
function Checkout() {
const variant = useVariant('checkout-redesign');
const isNewUi = useFeatureFlag('new-checkout');
return isNewUi ? <NewFlow /> : <OldFlow />;
}from experimently import ExperimentlyClient
client = ExperimentlyClient(api_key="YOUR_API_KEY")
# Get experiment variant
variant = client.get_variant(
"checkout-redesign", user_id
)
# Track conversion
client.track("purchase_completed", user_id,
properties={"revenue": 99.99})
# Feature flag with targeting
enabled = client.is_enabled(
"new-checkout", user_id,
context={"country": "US", "plan": "enterprise"}
)Enterprise-grade statistical methods — self-hosted, no price tag
| Feature | Experimently | Optimizely | Statsig* | Amplitude | LaunchDarkly |
|---|---|---|---|---|---|
| A/B / Multivariate Testing | ✅ | ✅ | ✅ | ✅ | ⚠️ Add-on |
| Feature Flags | ✅ | ✅ | ✅ | ✅ | ✅ |
| Sequential Testing (mSPRT) | ✅ | ✅ | ✅ | ✅ | ❌ |
| CUPED Variance Reduction | ✅ (binary + numeric) | ⚠️ Numeric only | ✅ | ✅ | ⚠️ Add-on |
| Multi-Armed Bandit | ✅ (3 algorithms) | ✅ | ✅ | ✅ | ❌ |
| Full Bayesian (BF10 + Monte Carlo) | ✅ | ❌ | ⚠️ Partial | ❌ | ❌ |
| Split URL Testing (Lambda@Edge) | ✅ | ✅ | ❌ | ❌ | ❌ |
| Interaction Detection | ✅ | ⚠️ Partial | ❌ | ❌ | ❌ |
| Mutual Exclusion Groups | ✅ | ⚠️ Partial | ✅ | ⚠️ Partial | ❌ |
| Global Holdout Group | ✅ | ✅ | ⚠️ | ❌ | ❌ |
| Dimensional Analysis | ✅ + Bonferroni | ⚠️ Partial | ✅ | ⚠️ Partial | ❌ |
| No-Code Experiment Wizard | ✅ | ✅ | ⚠️ | ✅ (Web) | ❌ |
| AI Experiment Design | ✅ (Claude + MCP) | ✅ (Opal AI) | ❌ | ❌ | ❌ |
| Warehouse Analytics | ✅ (3 warehouses) | ✅ | ✅ | ✅ (Snowflake) | ❌ |
| Jira / Salesforce / GitHub Integration | ✅ | ⚠️ Partial | ❌ | ❌ | ⚠️ Partial |
| Java SDK | ✅ (Spring Boot starter) | ✅ | ✅ | ⚠️ | ✅ |
| React SDK (hooks + SSR) | ✅ | ✅ | ✅ | ⚠️ | ✅ |
| SOC 2 / ISO 27001 Compliance Export | ✅ (HMAC-signed) | ✅ | ⚠️ Enterprise | ⚠️ Enterprise | ✅ |
| Audit Log + SSE Stream | ✅ | ⚠️ | ⚠️ | ❌ | ✅ |
| Deploy in Your AWS Account | ✅ | ❌ | ❌ | ❌ | ❌ |
| Pricing (entry) | 🎉 Free Preview | $36K+/year | Free → $150+/mo | Free → Custom | Free → $20K+/year |
* Statsig acquired by OpenAI for $1.1B in September 2025. Future roadmap subject to change. ⚠️ = Partial or enterprise-only feature.
Every feature is available — self-host on your own AWS account at no cost.
Self-hosted on your AWS account — you own your data
Sequential testing, CUPED, Bayesian stats, Java/React/JS SDKs, and AI design — all completely free. Deploy to your own AWS.
Free Preview · Self-hosted · All features included