A Python data-analysis SOP should separate data preparation, experimental inference and project reproduction, preserving inputs, methods and limitations at every stage. This guide uses a fictional book-discussion website's A/B test of an entry point to a recommended reading list. It first checks synthetic events, then estimates a conversion difference and studies reproducible methods from open-source repositories. The numbers are calculation examples, not business results or an automatic instruction to launch a variant.
Write an analysis contract and data dictionary
The analysis unit is the randomly assigned user, not the number of clicks. Define exposure, success, observation window, duplicate events and missing states. The synthetic table contains demonstration identifiers, group, exposure time, reading-list completion and batch. It contains no real personal information. Keep raw input read-only and separate cleaning copies, reports and execution records. Before an experiment, define the primary metric, smallest useful change, sample planning, stopping rule and guardrails. If analysis begins after the experiment ends, say which choices were made retrospectively rather than presenting exploration as a prior plan. An owner must confirm assignment, logging definitions and permissions; the agent cannot infer an entire design from column names.
Scenario one: preserve raw evidence while cleaning and exploring
Start with SkillPI Skill Finder, then inspect the selected revision, licence and dependencies. The verified Exploratory Data Analysis entry supports bounded structure, missingness and anomaly profiling of authorised local tables, with Python 3.11 or later for its core tools. It does not silently clean inputs, print identifying raw records or establish confirmatory or causal conclusions. Inference is therefore a separate stage with an explicit statistical method. Inspect encoding, headers, types, time zones, duplicate keys, missing groups and success values. Preserve a parsing error instead of silently dropping malformed rows to make a script finish. Every cleaning rule needs a reason and count. For example, retaining the first log by a stable event key requires a confirmed meaning of duplicate; two distinct events should not be collapsed because their fields look similar. The pandas missing-data guidance describes detection with methods such as isna. Missing is not zero, and blank strings need treatment under the input contract. Do not fill all missing observations with zero simply to enable a test. Compare missingness, sample counts and windows by group. Investigate assignment imbalance or tracking problems before declaring a winner. Deliver a quality register, transformation rules, before-and-after counts and a record that raw input was preserved. Distribution plots need units, coverage and exclusions. Treat outliers as review questions first. If reasonable handling alternatives change the result, report that sensitivity rather than retaining only the most favourable analysis.
Scenario two: define the estimand before selecting a significance test
The example asks for the difference B minus A in the proportion of assigned users completing the reading-list action within a fixed window. Repeated events from one user are not independent people. Cluster assignment needs a design that accounts for clusters. Continuous durations, proportions and rank-based questions are different targets; a calculator should not choose among t, chi-square and rank tests merely from appearance and then issue a launch verdict. The open-source statsmodels project has a defined statistical purpose, documentation and BSD-3-Clause licence, making it useful prior art for mature calculations. It is a Python library, not a linked SkillPI skill. Its two-independent-proportions test documentation states the independent-binomial-sample assumption. Record method, direction, significance level, version and arguments rather than relying on defaults that can change. In the synthetic example, group A has 100 successes among 1,000 users and B has 120 among 1,000. The rates are 10% and 12%. The absolute difference is 2 percentage points and the relative change is 20%; those are not interchangeable labels. Using the ordinary independent-sample Wald normal approximation with within-group binomial variances gives an illustrative 95% interval of about -0.74 to +4.74 percentage points for B minus A. This calculation is not a recommendation to use that approximation for every dataset. The proportion-difference interval documentation provides multiple methods. Small samples, extreme rates or complex designs need an appropriate method and review. Interpret intervals and tests under a consistent plan. The illustrative interval crosses zero, so a numerically higher B rate is not established evidence of improvement, and the result does not prove an effect of exactly zero either. Compare uncertainty with the predefined smallest useful effect, guardrails and experiment validity before deciding whether to retain a variant, gather more evidence or redesign. Do not repeatedly inspect p-values and stop at significance unless a suitable sequential procedure was planned. Multiple metrics and subgroup searches require appropriate treatment of multiplicity and clear exploratory labelling. An uncertain result should remain uncertain rather than being converted into a confident answer through retrospective power calculations.
Deliver an experiment report with distinct claims
State the question, assignment and observation units, definitions, quality problems, sample sizes, group rates, effect and interval, method and limitations. Distinguish an observation from a model-based estimate and from a decision recommendation. A statistical function does not understand implementation cost, user experience or an instrumentation failure, so a responsible owner reviews any launch proposal. The verified Writing Guidelines can review the technical explanation, runbook and limitations if the document and associated handbook are available. It does not validate statistical choices or turn stylistic acceptance into experiment certification. No suitable specialised A/B SkillPI entry was verified here, so unverified names from the source topic are not attached as recommendations. The report should include enough execution information for a second analyst to reproduce the selected method. Preserve the analysis version and any exclusions. If an exploratory cut suggests a new question, make it a future planned analysis rather than silently replacing the original primary outcome.
Scenario three: reproduce GitHub methods rather than copying conclusions
Select a learning project by question fit, data rights, maintenance, dependencies, tests and resource requirements rather than star count. Read README, licence, data dictionary, examples and check entry points before running anything. Map input to output. Execute a notebook from top to bottom in a clean environment so hidden files and stale cell state do not become unrecorded prerequisites. Pin a suitable release of statsmodels or the chosen method example in an isolated environment. Start with a small synthetic case before substituting authorised data. Record commands, dependencies, input digest, outputs and random settings where relevant. Downloading code is not evidence that its execution has been inspected or authorised; network, install and credential steps need a understood purpose. Compare the reading-list experiment's estimand with the repository example. If learning RFM-style grouping, first define recency, frequency and the relevant value field and window. That is descriptive segmentation, not causal evidence for this experiment. Do not automatically import customer spending fields or marketing conclusions. A dataset change requires revisiting units, missingness and assumptions, not only replacing a filename. The output is a reproduction note: original question, adopted portion, changes, matching results, failures and reasons. If reproduction fails, retain a minimal error and environment. A screenshot alone does not establish that the original author was wrong, and another project's test record does not establish that this project passed. This practice stage is also a chance to improve the SOP. If a hidden dependency was discovered, add it to the environment record. If a sample relied on an incompatible assumption, document why it was rejected. Reuse should preserve understanding rather than just produce a runnable-looking notebook.
Troubleshooting and frequently asked questions
What if text is garbled or columns have unexpected types? Check encoding, delimiter and the data dictionary before accepting implicit conversions that can damage identifiers. What if participant counts differ from log totals? Inspect units, duplication and windows instead of deleting rows until they match. What if a test produces an implausible result? Confirm that successes do not exceed sample size, groups satisfy the design and method prerequisites hold; stop when necessary data is missing. Can a beginner use the workflow? Begin with a synthetic case and explicit acceptance, while learning or obtaining review of design and interpretation. Must all three categories be installed? Choose by stage, beginning with Finder and verifying the target. Libraries and skills are different objects. Does a free package eliminate costs? Licensing, host plans, model usage and external services differ; removal also cannot undo generated files. Does non-significance mean no useful effect? No. Inspect the effect interval against the planned decision threshold. A completed SOP preserves raw input, makes cleaning traceable, states the estimand, reproduces the method and keeps conclusions consistent with limitations. Those are the conditions that make a result useful for the next discussion.