Abstract
Learning causal representations from data without access to ground-truth causal graphs remains a central challenge in representation learning. Implicit approaches—where the model learns causal dependencies without explicitly parameterizing the causal graph—offer a compelling advantage over explicit methods by avoiding optimization difficulties such as local minima associated with adjacency matrix estimation. However, existing implicit methods often assume access to hard interventions, which are rarely feasible in real-world scenarios. In contrast, soft interventions—more prevalent in practice—modify causal mechanisms without severing parental dependencies, introducing subtle, ambiguous effects that confound learning. To address this, we propose ICRL-SM, a novel method for Implicit Causal Representation Learning from soft interventions using a causal mechanism switch variable. This variable captures unwanted changes induced by soft interventions, enabling the model to focus on the necessary variations that reflect underlying causal structure. Our framework leverages a variational autoencoder trained on paired pre- and post-intervention samples, and is theoretically grounded under a set of assumptions that ensure identifiability of causal representations. Although some assumptions (e.g., Gaussianity of latent variables, diffeomorphic decoders) are strict, we show empirically that our method performs robustly even when these are violated—demonstrating strong results on both synthetic benchmarks and real-world image datasets. These findings highlight the potential of ICRL-SM to bridge the gap between theoretical identifiability and practical applicability, advancing causal representation learning under realistic conditions. The source code for this paper is available at: https://github.com/sshirahmad/ICRL
Data Availability
No datasets were generated or analysed during the current study.
References
Ahuja, K., Mahajan, D., Wang, Y., & Bengio, Y. (2023). Interventional causal representation learning. Proceedings of Machine Learning Research. International Conference on Machine Learning, ICML (Vol. 202, pp. 372–407). PMLR.
Bagi, S. S. G., Gharaee, Z., Schulte, O., & Crowley, M. (2023). Generative causal representation learning for out-of-distribution motion forecasting. Proceedings of Machine Learning Research. International Conference on Machine Learning, ICML (Vol. 202, pp. 31596–31612). PMLR.
Brehmer, J., De Haan, P., Lippe, P., & Cohen, T. S. (2022). Weakly supervised causal representation learning. Advances in Neural Information Processing Systems, 35, 38319–38331.
Buchholz, S., Rajendran, G., Rosenfeld, E., Aragam, B., Schölkopf, B., & Ravikumar, P. (2023). Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 36, 45419–62.
Charpentier, B., Kibler, S., & Günnemann, S. (2022). Differentiable DAG sampling. In: The Tenth International Conference on Learning Representations, ICLR April 25-29. OpenReview.net, Virtual Event.
Cooper, G. F., & Yoo, C. (2013). Causal Discovery from a Mixture of Experimental and Observational Data
Correa, J. D., Bareinboim, E., Larochelle, H., Ranzato, M., Hadsell, R., & Balcan, M. (2020). General transportability of soft interventions: Completeness results. In H. Lin (Ed.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. NeurIPS
Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Kazakos, E., Ma, J., Moltisanti, D., Munro, J., Perrett, T., Price, W., & Wray, M. (2022). Rescaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100. International Journal of Computer Vision, 130(1), 33–55. https://doi.org/10.1007/s11263-021-01531-2
Deitke, M., VanderBilt, E., Herrasti, A., Weihs, L., Ehsani, K., Salvador, J., Han, W., Kolve, E., Kembhavi, A., & Mottaghi, R. (2022). Procthor: Large-scale embodied ai using procedural generation. Advances in Neural Information Processing Systems, 35, 5982–5994.
Eastwood, C., & Williams, C. K. I. (2018). A framework for the quantitative evaluation of disentangled representations. 6th International Conference on Learning Representations, ICLR
Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., Mohamed, S., & Lerchner, A. (2017). beta-vae: Learning basic visual concepts with a constrained variational framework. 5th International Conference on Learning Representations, ICLR
Immer, A., Schultheiss, C., Vogt, J. E., Schölkopf, B., Bühlmann, P., & Marx, A. (2023). On the identifiability and estimation of causal location-scale noise models. Proceedings of Machine Learning Research. International Conference on Machine Learning, ICML 2023, 23–29 July (Vol. 202, pp. 14316–14332). PMLR.
Jaber, A., Kocaoglu, M., Shanmugam, K., & Bareinboim, E. (2020). Causal discovery from soft interventions with unknown targets: Characterization and learning. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. NeurIPS
Kaddour, J., Lynch, A., Liu, Q., Kusner, M. J., & Silva, R. (2022). Causal machine learning: A survey and open problems. arXiv preprint arXiv:2206.15475.
Kivva, B., Rajendran, G., Ravikumar, P. K., & Aragam, B. (2022). Identifiability of deep generative models under mixture priors without auxiliary information. UAI 2022 Workshop on Causal Representation Learning
Kügelgen, J., Besserve, M., Wendong, L., Gresele, L., Kekić, A., Bareinboim, E., Blei, D. M., & Schölkopf, B. (2023). Nonparametric identifiability of causal representations from unknown interventions. Advances in Neural Information Processing Systems, 36, 48603–38.
Kügelgen, J., Sharma, Y., Gresele, L., Brendel, W., Schölkopf, B., Besserve, M., & Locatello, F. (2021). Self-supervised learning with data augmentations provably isolates content from style. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. W. Vaughan (Eds.), Advances in Neural Information Processing Systems (Vol. vol. 34, pp. 16451–16467). Curran Associates, Inc.
Lachapelle, S., Rodríguez, P., Sharma, Y., Everett, K., Priol, R.L., & Lacoste, A., Lacoste-Julien, S. (2022). Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA. In: 1st Conference on Causal Learning and Reasoning, CLeaR. Proceedings of Machine Learning Research, vol. 177, pp. 428–484. PMLR, Sequoia Conference Center.
Li, R., Pu, Y., Li, Z., Wu, C., Xie, H., & Lian, D. (2025). Invariant representation learning via decoupling style and spurious features. Machine Learning, 114(2), 37.
Lippe, P., Cohen, T., & Gavves, E. (2022). Efficient neural causal discovery without acyclicity constraints. In: The Tenth International Conference on Learning Representations, ICLR April 25-29. OpenReview.net, Virtual Event.
Lippe, P., Magliacane, S., Löwe, S., Asano, Y. M., Cohen, T., & Gavves, S. (2022). CITRIS: causal identifiability from temporal intervened sequences. Proceedings of Machine Learning Research. International Conference on Machine Learning, ICML (Vol. 162, pp. 13557–13603). PMLR.
Liu, C., Sun, X., Wang, J., Tang, H., Li, T., Qin, T., Chen, W., & Liu, T.-Y. (2021). Learning causal semantic representation for out-of-distribution prediction. NeurIPS 2021, December 6–14. Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 (Vol. 34, pp. 6155–6170). Virtual Event: Curran Associates Inc.
Liu, Y., Alahi, A., Russell, C., Horn, M., Zietlow, D., Schölkopf, B., & Locatello, F. (2023). Causal triplet: An open challenge for intervention-centric causal representation learning. Proceedings of Machine Learning Research. Conference on Causal Learning and Reasoning, CLeaR 2023, 11–14 April (Vol. 213, pp. 553–573). Amazon Development Center. PMLR.
Liu, Y., Cadei, R., Schweizer, J., Bahmani, S., & Alahi, A. (2022). Towards robust and adaptive motion forecasting: A causal representation perspective. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, June 18-24, 2022, pp. 17060–17071. IEEE. https://doi.org/10.1109/CVPR52688.2022.01657
Locatello, F., Poole, B., Rätsch, G., Schölkopf, B., Bachem, O., & Tschannen, M. (2020). Weakly-supervised disentanglement without compromises. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020. Proceedings of Machine Learning Research, vol. 119, pp. 6348–6359. PMLR, Virtual Event.
Lu, C., Wu, Y., Hernández-Lobato, J. M., & Schölkopf, B. (2022). Invariant causal representation learning for out-of-distribution generalization. The Tenth International Conference on Learning Representations, ICLR
Ni, H., Wang, T.-Z., Tao, H., Huang, X., & Hou, C. (2025). Gradient-based causal discovery with latent variables. Machine Learning, 114(2), 46.
Pearl, J. (2005). Causality, cambridge university press (2000). Artificial Intelligence. 169(2), 174–179.
Pearl, J., Glymour, M., & Jewell, N. P. (2016). Causal Inference in Statistics: A Primer (2nd ed.). John Wiley & Sons.
Perry, R., Kügelgen, J., & Schölkopf, B. (2022). Causal discovery in heterogeneous environments under the sparse mechanism shift hypothesis. In: Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022.
Schölkopf, B. (2022). Probabilistic and Causal Inference: The Works of Judea Pearl (pp. 765–804) Causality for machine learning.
Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., & Bengio, Y. (2021). Toward causal representation learning. Proceedings of the IEEE, 109(5), 612–634. https://doi.org/10.1109/JPROC.2021.3058954
Shen, X., Liu, F., Dong, H., Lian, Q., Chen, Z., & Zhang, T. (2022). Weakly supervised disentangled generative causal representation learning. Journal of Machine Learning Research, 23, 241–124155.
Squires, C., Seigal, A., Bhate, S., & Uhler, C. (2023). Linear Causal Disentanglement via Interventions
Varici, B., Acarturk, E., Shanmugam, K., Kumar, A., & Tajer, A. (2023). Score-based Causal Representation Learning with Interventions. arXiv preprint arXiv:2301.08230
Varici, B., Acartürk, E., Shanmugam, K., Kumar, A., & Tajer, A. (2025). Score-based causal representation learning: Linear and general transformations. Journal of Machine Learning Research, 26(112), 1–90.
Wendong, L., Kekić, A., Kügelgen, J., Buchholz, S., Besserve, M., Gresele, L., & Schölkopf, B. (2023). Causal component analysis. In: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023.
Yang, M., Liu, F., Chen, Z., Shen, X., Hao, J., & Wang, J. Causalvae: Disentangled representation learning via neural structural causal models. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, June 19-25, 2021, pp. 9593–9602. Computer Vision Foundation / IEEE, Virtual Event (2021). https://doi.org/10.1109/CVPR46437.2021.00947
Yang, S., Yu, K., Cao, F., Liu, L., Wang, H., & Li, J. (2021). Learning causal representations for robust domain adaptation. IEEE Transactions on Knowledge and Data Engineering,. https://doi.org/10.1109/TKDE.2021.3119185
Yu, K., Guo, X., Liu, L., Li, J., Wang, H., Ling, Z., & Wu, X. (2020). Causality-based feature selection: Methods and evaluations. ACM Computing Surveys. https://doi.org/10.1145/3409382
Yu, Y., Chen, J., Gao, T., & Yu, M. (2019). DAG-GNN: DAG structure learning with graph neural networks. In: Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019. Proceedings of Machine Learning Research, vol. 97, pp. 7154–7163. PMLR.
Zhang, J., Greenewald, K.H., Squires, C., Srivastava, A., Shanmugam, K., & Uhler, C. (2023). Identifiability guarantees for causal disentanglement from soft interventions. In: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023.
Zheng, X., Aragam, B., Ravikumar, P., & Xing, E. P. (2018). Dags with NO TEARS: continuous optimization for structure learning. Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems NeurIPS (pp. 9492–9503)
Author information
Authors and Affiliations
Contributions
SSGB developed and proved the theoretical framework. With guidance from ZG, SSGB designed, implemented the experiments, and analyzed the results. ZG generated all figures and tables. SSGB and ZG jointly wrote the manuscript. OS contributed to the theoretical proofs. OS and MC reviewed and revised the manuscript. SSGB and ZG jointly revised the manuscript and responded to the reviewers comments.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Editor: Henry Gouk.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix
1.1 Identifiability Assumptions
Learning identifiable causal representations from soft interventions is inherently ill-posed without suitable assumptions. We introduce a set of assumptions that make identification feasible in our implicit setting and briefly discuss their necessity and potential for relaxation.
Assumption A.1
(Atomic Interventions) Let \({\mathcal {I}}=\{1,2,...,n\}\) denote the intervention target set. Each sample \((x,\tilde{x},i)\) (\(i \in {\mathcal {I}})\) involves an intervention on only one causal variable \(Z_i\).
-
Rationale: Ensures attribution of change to a single source.
-
Relaxation: May be relaxed using multi-target extensions with additional assumptions on sparsity.
Assumption A.2
(Complete Intervention Target Set) All causal variables should be intervened on, i.e., \({\mathcal {I}}=\{1,2,...,n\}\).
-
Rationale: Guarantees full identifiability coverage.
-
Relaxation: Identifiability may still hold for partial targets with structural priors or additional data.
Assumption A.3
(Known Targets) The target of intervention t is known for each sample.
-
Rationale: Allows accurate conditioning on the affected variable.
-
Relaxation: Can be estimated using techniques from Brehmer et al. (2022).
Assumption A.4
(Counterfactual Exogenous Variables) Only the exogenous variable corresponding to the target changes:
-
Rationale: Guarantees only the targeted exogenous variable changes, enabling disjoint support in \((E,\tilde{E})\) and a well-defined intervention map.
Assumption A.5
(Sufficient Action Variability) For all distinct intervention targets \(i \ne j \in {\mathcal {I}}\), the distributions of interventional pairs \((z,\tilde{z})\) under i and j are mutually exclusive:
-
Rationale: Ensures interventions create distinct, non-overlapping outcomes in latent space for identifiability.
Assumption A.6
(Diffeomorphic Decoders and Mechanisms) The decoder g and solution functions \(s_i\) are smooth, invertible functions.
-
Rationale: Prevents information loss and preserves topology.
-
Relaxation: Empirically, identifiability holds even when this assumption is violated in real datasets.
Assumption A.7
(Observability of V) The soft intervention effect term in the Taylor expansion of \(s_i\) is observable or can be approximated from \(\tilde{x} - x\) when g is linear (More details in Section 4.3).
-
Rationale: Enables estimation of causal mechanism shifts.
-
Relaxation: For nonlinear decoders, other augmentations or learned proxies may be used instead. Empirically, identifiability holds even when this assumption is violated in real datasets.
Assumption A.8
(Gaussianity of Latents) Causal and exogenous variables follow a multivariate normal distribution.
-
Rationale: Simplifies identifiability proof in Lemma 1.
-
Relaxation: Empirically, identifiability holds even when this assumption is violated in real datasets.
Assumption A.9
(Affinity) For each i, the solution function \(\tilde{s}_i: {\mathbb {R}}\times {\mathbb {R}}^{n-1} \times {\mathbb {R}}^n \rightarrow {\mathbb {R}}\), defined as \(s_i(\tilde{e}_i;e_{/i},v)\) in Section 4.6, is an affine function of its arguments, which satisfies:
where \(f_i:{\mathbb {R}}\rightarrow {\mathbb {R}}\) is a diffeomorphism, \(h_{1,i}:{\mathbb {R}}^{n-1} \times {\mathbb {R}}^n \rightarrow {\mathbb {R}}\), and \(h_{2,i}:{\mathbb {R}}^{n-1} \times {\mathbb {R}}^n \rightarrow {\mathbb {R}}\) are arbitrary functions.
-
Rationale: Simplifies identifiability proof (Lemma 1).
-
Relaxation: Empirically, identifiability holds even when this assumption is violated in real datasets.
Identifiability Theorem
Theorem 2
(Identifiability of latent causal models.) Let \({\mathcal {M}}=({\mathcal {A}}, {\mathcal {X}}, g, {\mathcal {I}})\) and \({\mathcal {M}}'=({\mathcal {A}}', {\mathcal {X}}, g', {\mathcal {I}})\) be two LCMs with shared observation space \({\mathcal {X}}\) and shared intervention targets \({\mathcal {I}}\). Suppose the assumptions in Appendix 9. are satisfied. Then the following statements are equivalent:
-
1.
Two LCMs \({\mathcal {M}}\) and \({\mathcal {M}}'\) assign the same likelihood to interventional and observational data i.e., \(p^{{\mathcal {X}}}_{{\mathcal {M}}}(x,\tilde{x}) = p^{{\mathcal {X}}}_{{\mathcal {M}}'}(x, \tilde{x})\).
-
2.
\({\mathcal {M}}\) and \({\mathcal {M}}'\) are disentangled, that is \({\mathcal {M}} \sim _r {\mathcal {M}}'\) according to Definition 6.
Proof
We will proceed to prove the equivalence between statements 1 and 2 by showing the implication is true in both forward and reverse directions.
1.1 Forward Direction: \({\mathcal {M}} \sim _r \mathcal {M'} \Rightarrow p^{{\mathcal {X}}}_{{\mathcal {M}}}(x,\tilde{x}) = p^{{\mathcal {X}}}_{\mathcal {M'}}(x, \tilde{x})\)
This direction is fairly straightforward. According to Definition 6, the fact that \(M \sim _r M'\) implies that \(\phi _{{\mathcal {E}}}\) is measure preserving. Thus:
Furthermore, considering that ancestry is preserved, \(\phi _{{\mathcal {Z}}}\) is measure preserving, and that causal variables are obtained from their ancestral exogenous variables in implicit models, we have:
Since models are trained to maximize the log likelihood of \(p(x, \tilde{x}, \tilde{x} - x)\) and the latent spaces in M and \(M'\) have the same distribution, the decoders should yield the same observational distributions as:
1.2 Reverse Direction: \(p^{{\mathcal {X}}}_{{\mathcal {M}}}(x,\tilde{x}) = p^{{\mathcal {X}}}_{\mathcal {M'}}(x, \tilde{x}) \Rightarrow {\mathcal {M}} \sim _r \mathcal {M'}\)
Let’s define \(\phi _{{\mathcal {E}}}=g'^{-1} \circ g: {\mathcal {E}} \rightarrow \mathcal {E'}\). Since we can express \(e=s^{-1}(z)\), we can now define \(\phi _{{\mathcal {Z}}}\) as:
Therefore, we have:
Because g and \(g'\) are diffeomorphisms (Assumption A.6), \(\phi _{{\mathcal {E}}}\) is a diffeomorphism as well. Furthermore, since \(p^{{\mathcal {X}}}_{{\mathcal {M}}}=p^{{\mathcal {X}}}_{\mathcal {M'}}\) and \(\phi _{{\mathcal {E}}}\) is a diffeomorphism, then:
Consequently, \(\phi _{{\mathcal {E}}}\) is measure-preserving. Similarly, \(\phi _{{\mathcal {Z}}}\) is measure-preserving as well since causal mechanisms are diffeomorphic (Assumption A.6).
1.2.1 Step 1: Identical Correspondence of Edges and Nodes
We assume interventions are atomic in Assumption A.1. Let’s define the set U for atomic interventions \(i \ne j \in {\mathcal {I}}\) as:
By Assumption A.4, for any pair \((e,\tilde{e}) \sim p(E,\tilde{E} \mid I=i)\), only the component \(e_i\) changes between e and \(\tilde{e}\) while all others remain fixed: \(e_k=\tilde{e}_k \quad \forall k\ne i\) and \(e_i \ne \tilde{e}_i\).
Similarly, for \(j \ne i\), any sample from \(p(E,\tilde{E}\mid I=j)\) satisfies \(e_k=\tilde{e}_k \quad \forall k\ne j\), and \(e_j\ne \tilde{e}_j\).
Thus, any pair \((e,\tilde{e})\) from \(supp\, p(E,\tilde{E}\mid I=i)\) must satisfy \(e_i \ne \tilde{e}_i\), but \(e_j=\tilde{e}_j\), while pairs from \(supp\, p(E,\tilde{E}\mid I=j)\) satisfy the reverse. Therefore, no pair can simultaneously satisfy both constraints, and the supports are disjoint:
Under Assumption A.2, we can write the marginal distribution \(p_{{\mathcal {M}}}^{{\mathcal {E}}}(E, \tilde{E})\) as:
According to Equation 38, we can say that \(p_{{\mathcal {M}}}^{{\mathcal {E}}}(E, \tilde{E})\) is a discrete mixture of non-overlapping distributions \(p_{{\mathcal {M}}}^{{\mathcal {E}}, {\mathcal {I}}}(E, \tilde{E}\mid I=i)\). Similarly, we can say that \(p_{\mathcal {M'}}^{{\mathcal {E}}'}(E', \tilde{E}')\) is a discrete mixture of non-overlapping distributions. Since \(\phi _{{\mathcal {E}}}\) is measure preserving, \(p_{{\mathcal {M}}}^{{\mathcal {E}}}(E, \tilde{E})\) = \(p_{\mathcal {M'}}^{{\mathcal {E}}'}(E', \tilde{E}')\). Therefore,
It can be concluded that as \(\phi _{{\mathcal {E}}}\) must map between the conditional distributions, there exists a bijection that also induces a permutation \(\psi : [n] \rightarrow [n]\).
Under Assumption A.3, we have a shared domain \({\mathcal {I}}\) in \({\mathcal {M}}\) and \({\mathcal {M}}'\). Because the mixture over \((E,\tilde{E})\) is equal across models (Equation 40), the conditional components must match under the same index. Therefore, the permutation \(\psi\) is the identity transformation.
In the latent space \({\mathcal {Z}}\), it is crucial that interventions induce sufficient variability so that the conditional interventional distributions \(p_{{\mathcal {M}}}^{{\mathcal {Z}},{\mathcal {I}}}(Z,\tilde{Z}\mid I=i)\) are disjoint for different intervention targets i.
Because soft interventions modify the underlying causal mechanisms rather than directly fixing variables, the value of causal variable \(\tilde{z}\) depends on all ancestral exogenous variables affecting the intervened variable. Since these exogenous variables may influence multiple causal variables, insufficiently distinct mechanism changes can cause overlaps in the distributions associated with different interventions. This overlap arises because similar shifts in ancestral variables produce similar latent outcomes, blurring the distinctions between interventions.
By assuming sufficient variability (Assumption A.5), we ensure that each intervention \(i \in {\mathcal {I}}\) produces a unique, non-overlapping distribution in \({\mathcal {Z}} \times {\mathcal {Z}}\). This property mirrors the non-overlapping supports we have assumed in the exogenous space \({\mathcal {E}}\).
Consequently, there exists a permutation mapping between the interventional distributions \(p_{{\mathcal {M}}}^{{\mathcal {Z}},{\mathcal {I}}}(Z,\tilde{Z} \mid I=i)\) and \(p_{{\mathcal {M}}'}^{\mathcal {Z'},{\mathcal {I}}}(Z',\tilde{Z}'\mid I=i)\). Since the intervention targets are known and shared, this permutation reduces to the identity mapping, preserving the correspondence between interventions across models.
The effect of soft intervention with known targets on these conditional distributions is shown in Figure 6.
1.2.2 Step 2: Component-Wise \(\phi _{{\mathcal {Z}}}\)
We are going to propose a Lemma and prove it before proceeding to the next step in a proof.
Lemma 1
The transformation \(\phi _{{\mathcal {Z}}}: {\mathcal {Z}} \rightarrow \mathcal {Z'}\) between the causal variable of two LCMs \({\mathcal {M}}\) and \(\mathcal {M'}\) defined in Definition 6 is a component-wise transformation, if \(\, \forall i\ne j \quad \tilde{E'_i} \perp \!\!\!\perp \tilde{E'_j} \mid E', V'\).
proof:
Under Assumption A.9 and Assumption A.7 (V is observed and shared between \({\mathcal {M}}\) and \({\mathcal {M}}'\)) we can write:
for simplicity let’s just say \({\tilde{Z}}'_i = h'_{1,i} \cdot f'_i({\tilde{e}}'_i) + h'_{2,i}\).
Given \({\tilde{E}}'_i \perp \!\!\!\perp {\tilde{E}}'_j \mid E', V\), we compute:
Similarly:
and thus,
Hence:
Now, under Assumption A.8 we can write:
Let \(\phi _{{\mathcal {E}}}=g'^{-1} \circ g: {\mathcal {E}} \rightarrow \mathcal {E'}\), where g and \(g'\) are the decoders in \({\mathcal {M}}\) and \({\mathcal {M}}'\), respectively. As per Assumption A.6, both decoders are diffeomorphic, so \(\phi _{{\mathcal {E}}}\) is also a diffeomorphism. Let \(s({\tilde{e}}; e, v)=[s_1({\tilde{e}}_1; e_{/1}, v), s_2({\tilde{e}}_2; e_{/2}, v), \ldots , s_n({\tilde{e}}_n; e_{/n}, v)]\) denote the set of all solution functions in the post-intervention system, which are diffeomorphic by Assumption A.6. Furthermore, let’s define \(\tilde{Z}=(\tilde{Z}_1,\ldots ,\tilde{Z}_n)\in {\mathbb {R}}^n\). Then:
We use the Pigeonhole principle to arrive at the conclusion. Let’s define holes as \(\tilde{Z}=(\tilde{Z}_1,\ldots ,\tilde{Z}_n)\in {\mathbb {R}}^n\) and pigeons as the dependencies of \(\tilde{Z}'=(\tilde{Z}'_1,\ldots , \tilde{Z}'_n)\in {\mathbb {R}}^n\) - for each i, let the \(\tilde{Z}'_i\) depend on some subset of \(\tilde{Z}\). So if \(\tilde{Z}'_i\) depends on \(k_i \ge 1\) number of variables in \(\tilde{Z}\), we say that it occupies \(k_i\) holes. The reason for \(k_i \ge 1\) is that \({\mathcal {M}}\) and \({\mathcal {M}}'\) have shared observation space.
Suppose for contradiction that at least one \(\tilde{Z}'_i\) depends on two or more variables \(\tilde{Z}\); say \(\tilde{Z}_j\) and \(\tilde{Z}_k\), with \(j\ne k\). So \(k_i \ge 2\).
To satisfy the conditional independence \(\tilde{Z}'_i \perp \!\!\!\perp \tilde{Z}'_r \mid \tilde{Z}, E, V\), no two \(\tilde{Z}'_i, \tilde{Z}'_r\) can share any common dependency on the same variable from \(\tilde{Z}\); because shared dependency creates conditional dependence given \(\tilde{Z}\), violating the assumption.
Now we apply the pigeonhole principle:
There are n holes and there are n pigeons, each requiring at least one hole. At least one \(\tilde{Z}'_i\) requires at least two holes. In order to not violate the assumption, we would need more than n holes in total which is a contradiction. Hence, \(\phi _{{\mathcal {Z}}}\) must be component-wise.
According to Lemma 1, in order to prove that \(\phi _{{\mathcal {Z}}}\) is a component-wise transformation, we need to prove that \(\forall i\ne j \quad {\tilde{E}}'_i \perp \!\!\!\perp {\tilde{E}}'_j \mid E', V'\). In implicit modeling we do not know the parents of each causal variable. Since \(E'_i\) is a known parent of \(\tilde{Z'_i}\). The mean of the conditional distribution of \(p^{{\mathcal {Z}}}_{{\mathcal {M}}'}(\tilde{Z}'_i|E'=e', V'=v')\) can be calculated as:
where \(\rho\) and \(\sigma\) are the correlation coefficient and variance of the random variables, respectively. On the other hand, we model \(\tilde{Z'_i}\) using switch mechanisms as:
By using Taylor’s expansion we can write Equation 42 as:
Consequently, we can write the following equality from Equation 41:
Under hard intervention, structural equations are replaced by constants; thus, the right-hand side contains no dependence on ancestor variables. This implies:
In contrast, under soft intervention, parent variables can still influence \(\tilde{Z}'_i\) through \(s'_i\), which introduces dependencies if we did not have the \(R_i\) term:
By introducing the causal mechanism switch variable \(V'\), and assuming it is observed, we isolate the soft intervention effects into an additive term \(R_i\). This enables the model to learn exogenous variables \(\tilde{E}'_i\) that satisfy:
which is consistent with the standard assumption that exogenous variables are mutually independent. Consequently:
1.2.3 Step 3: Component-Wise \(\phi _{{\mathcal {E}}}\)
Using the result from previous step that \(\phi _{{\mathcal {Z}}}\) is a component-wise transformation, the string diagrams for connections between E and \(E'\) will be as shown in Figure 7. \(\phi _{{\mathcal {E}}}(e)_{i'} = (s'^{-1} \circ \phi _{{\mathcal {Z}}} \circ s)(e)_{i'}\) will only depend on \(E_{anc_i}\), where \(anc_i\) is the indices of ancestors of variable i.
Because:
Please note that the arguments of the solution functions presented above reflect their dependencies, rather than the precise inputs used in the implementation.
The first equality in Figure 7 follows from the definition of \(\phi _{{\mathcal {E}}_i}\). The second equality holds when we first apply \(\phi _{{\mathcal {Z}}_A}\) and then apply the causal mechanisms. It can be concluded from the most right-hand side diagram in Figure 7 that the transformation from \({\mathcal {E}}_i \times {\mathcal {E}}_A \rightarrow {\mathcal {E}}'_i\) is constant in \({\mathcal {E}}_A\). Therefore, \(\phi _{{\mathcal {E}}_i}\) is a component-wise transformation.
Experiments
This section contains additional details about ICRL-SM design architectures and experiments settings.
1.1 Architecture Design
Based on the ICRL-SM architecture (Figure 3), we design a location-scale solution function (Equation 24), where the \(\operatorname {loc}_i\), \(\operatorname {scale}_i\), and \(h_i\) networks are all fully connected. Each network consists of two layers with 64 hidden units per layer and ReLU activations. The encoder and decoder for latents E and \({\tilde{E}}\) are shared, while a separate encoder-decoder pair with the same architecture is used for latent V.
For synthetic dataset experiments, encoders and decoders are fully connected networks with two hidden layers of 64 units each. For the Causal-triplet datasets, we employ ResNet backbones. All baseline models use the same encoder-decoder architectures to ensure fair comparison. Specifically:
ResNet-50 experiments (Table 7,Table 10): ResNet-50 encoder and decoder, with classifiers having one hidden layer of 64 units, predict actions and objects.
ResNet-18 experiments (Table 9): ResNet-18 encoder and decoder, with classifiers of two hidden layers and 2 units each, predict actions and objects.
1.2 Training
To enforce the condition described in Equation 23 for \(i \notin {\mathcal {I}}\), we assign the post-intervention exogenous variables the same value as the pre-intervention exogenous variables. In mathematical terms, this translates to \(\forall i \notin {\mathcal {I}}\), we set \(\tilde{e}_i = e_i\).
In our experiments, we do not pretrain the networks, however, for the baseline models we follow the training procedure in Brehmer et al. (2022). In addition, we impose a consistency constraint to encourage the encoder and decoder to act as approximate inverses of each other. The consistency regularizer is defined as:
where \({\hat{x}}\) denotes the reconstructed samples produced by the decoder.
Batch Size. In all experiments, the batch size is 64.
Learning Rate. For optimization, we use the Adam optimizer with default hyperparameters. In synthetic experiments (Table 5), the learning rate decays from \(3\times 10^{-4}\) to \(1\times 10^{-8}\) using a cosine scheduler. For the Causal-triplet experiments (Table 7), the learning rate decays from \(2\times 10^{-3}\) to \(1\times 10^{-8}\), while for the ResNet-18 experiments (Table 9) it decays from \(1\times 10^{-4}\) to \(1\times 10^{-8}\).
Epoch. Training schedules are 400 epochs for Table 7, 2000 epochs for Table 9, and 100 epochs for the synthetic experiments (Table 5). For Table 9, graph parameters of the explicit models are frozen after 1000 epochs.
Compute Infrastructure. All models are trained on NVIDIA GeForce RTX 4090 GPUs. Each Causal-Triplet experiment requires 3–8 hours, while each synthetic experiment requires 2–3 hours. We save the model weights with the best validation loss and use them for evaluation on the test data.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shirahmad Gale Bagi, S., Gharaee, Z., Schulte, O. et al. Modeling Soft Intervention Effects for Implicit Causal Representation Learning. Mach Learn 115, 199 (2026). https://doi.org/10.1007/s10994-026-07144-5
Received:
Revised:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s10994-026-07144-5