Robust and efficient copulas

Setup and overview

Copula-based probability models \[ \underbrace{H(x_1,\ldots, x_M)}_{\text{multivariate c.d.f.}}=\underbrace{C\big(\overbrace{F_1(x_1),\ldots,F_M(x_M)}^{\text{univariate marginal c.d.f.'s}}\big)}_{\text{copula function}} \quad \Leftrightarrow \quad \underbrace{h(x_1,\ldots, x_M)}_{\text{multivariate p.d.f.}}=\underbrace{c\big(F_1(x_1),\ldots,F_M(x_M)\big)}_{\text{copula density}}\prod_{m=1}^M \underbrace{f_t(x_m)}_{\text{marginal p.d.f.}} \]

Copulas are multivariate distributions with uniform marginals.

Start with \(M=2\); assume correctly specified marginals, known up to \(\beta\): \[ \beta\in\mathbb{R}^p \quad\text{ collects all distinct parameters of marginals}\]

Dependence is nuisance: \[\gamma\in\mathbb{R}^{d_\gamma} \quad\text{ collects nuisance parameters of copula}\]

Log-likelihood for a sample of size \(N\) is \[\ln L(\beta; \gamma) = \sum_{n=1}^N \Big[ \overbrace{\ln f_1 (x_{1n}; \beta) + \ln f_2 (x_{2n}; \beta)}^{\text{Quasi-MLE (QMLE)}} + \overbrace{\ln c\big(F_1 (x_{1n}; \beta) ; F_2 (x_{2n}; \beta);\gamma\big)}^{\text{nuisance}}\Big]\]

Likelihood-based moment conditions

\[\ln L(\beta; \gamma) = \sum_{n=1}^N \Big[ \underbrace{\ln f_1 (x_{1n}; \beta)}_{\ln f_{1n}(\beta)} + \underbrace{\ln f_2 (x_{2n}; \beta)}_{\ln f_{2n}(\beta)} + \underbrace{\ln c\big(F_1 (x_{1n}; \beta) ; F_2 (x_{2n}; \beta);\gamma\big)}_{ \ln c_{n}(\beta, \gamma)}\Big]\]

Full MLE: \[ \max_{\beta,\gamma}\log L(\beta,\gamma) = \max_{\beta,\gamma} \sum_{n=1}^N \left[ \ln f_{1n}(\beta) + \ln f_{2n}(\beta) + \ln c_n(\beta,\gamma) \right]. \]

Suppose copula is correctly specified too:

\[\exists \beta_o, \gamma_o : \underbrace{\mathbb{E}\!\left[\nabla_\beta \ln f_1(\beta_o)\right]}_{=0} + \underbrace{\mathbb{E}\!\left[\nabla_\beta \ln f_2(\beta_o)\right]}_{=0} + \mathbb{E}\!\left[\nabla_\beta \ln c(\beta_o,\gamma_o)\right] =0, \] \[ \mathbb{E}\!\left[\nabla_\gamma \ln c(\beta_0,\gamma_o)\right]=0. \]

In search of the Holy Grail

\[(*)\quad\left\{\begin{matrix} \mathbb{E}\!\nabla_\beta \ln k(\beta_0,\gamma_0^k)\ne 0 \\ \mathbb{E}\!\nabla_\gamma \ln k(\beta_0,\gamma_0^k)\ne 0 \end{matrix} \right. \]

A misspecified copula \(k\) generally invalidates \((*)\) \(\Rightarrow\widehat\beta\) inconsistent.

Part 1. Parametrics: \(\dim(\gamma)<\infty\)

Are there robust and efficiency improving copula families?

Part 2. Semiparametrics: \(\dim(\gamma) = \infty\)

Can a consistent nonparametric estimator of \(c\) be efficiency improving?

Part 3. Machine learning: semiparametric with penalisation of \(c\) and orthogonolised moments.

Can robust machine learning help?

Available parametric estimators

FMLE = GMM based on \(\left[\begin{array}{c}(a)+(b)+(c)\\(d)\end{array}\right]\) versus QMLE = GMM based on (\(a\))+(\(b\))

When are copulas robust?

Trivial answer: The independence copula \[K(u, v) = u\cdot v \Rightarrow \mathbb{E}\nabla_{(\beta, \gamma)} \ln k(\beta_o,\gamma_o) =0\]

Non-trivial answer: Unknown. Problem specific.

E.g., problems where \(\beta\) = the mean of \(F_1\) and \(F_2\):

Symmetry

Use any radially symmetric copula with any \(\gamma\) and still get consistent \(\widehat{\beta}\)

When are copulas informative about \(\beta\)?

Whenever moments (c)-(d) are non-redundant for estimation of \(\beta\) given moments (a)-(b)

Digression on redundancy

Consider \(\qquad\mathbb{E} g(Z,\theta_o) = \left[\begin{array}{cc} \mathbb{E}g_1(Z,\theta_o) \\ \mathbb{E}g_2(Z,\theta_o) \end{array} \right]=0\)

Only \(g_1\) identifies \(\theta\).

\(g_2\) is redundant given \(g_1\) \(\Leftrightarrow\) GMM based on only \(g_1\) is as efficient as GMM based on \((g_1,g_2)\).

Notation: \(\quad \mathbb{D}_1=\left[\begin{array}{c}\mathbb{E}\nabla_\theta g_1 \\ \mathbb{E}\nabla_\theta g_2\end{array}\right] = \left[\begin{array}{c}\mathbb{D}_1 \\ \mathbb{D}_2\end{array}\right]\qquad \mathbb{C}=\left[\begin{array}{cc}\mathbb{E}g_1 g_1'&\mathbb{E}g_1 g_2' \\ \mathbb{E}g_2 g_1'&\mathbb{E}g_2 g_2'\end{array}\right] =\mathbb{V}\left[\begin{array}{c} g_{1}\\ g_{2}\end{array}\right] = \left[\begin{array}{cc} \mathbb{C}_{11}& \mathbb{C}_{12}\\ \mathbb{C}_{21}&\mathbb{C}_{22}\end{array}\right]\)

\(\mathbb{V}_{1}\) = asy variance of (optimal) GMM estimator of \(\theta_o\) based on \(g_1\)

\(\mathbb{V}_{1\&2}\) = asy variance of (optimal) GMM estimator of \(\theta_o\) based on \((g_1, g_2)\)

Digression on redundancy, cont’d

End of digression

Redundancy of true copula

Suppose \(\ln c\) is correctly specified and \(\gamma = \gamma_o\) is known. Relevant moments:

What are \(\mathbb{D}\) and \(\mathbb{C}\)? When is \(\mathbb{D}_2=\mathbb{C}_{21}\mathbb{C}^{-1}_{11}\mathbb{D}_1\)?

Redundancy of estimated copula

• correctly specified, unknown \(\gamma\)

• incorrectly specified but robust, unknown \(\gamma\)

Examples

Point: copulas are generally efficiency-increasing, i.e. non-redundant, but how much more efficiency?

How much more efficiency?

Asy Relative Efficiency: \[\text{ARE} = \frac{\mathbb{V}_\text{FMLE}}{\mathbb{V}_\text{QMLE}}\]

Semiparametric setup

Note: redundancy of parametric copulas is rare; it guarantees robustness but not vice versa.

Still possible to find the Holy Grail?

Semiparametrics: estimate \(\beta\) along with the unknown copula function.

Consider Sieve MLE:

\[ \ln L(\beta, \gamma) =% \sum_{i=1}^N l_i(\beta, \gamma)= \sum_{i=1}^N [\ln f_1 (x_{1i}; \beta) + \ln f_2 (x_{2i}; \beta) + \ln \gamma(F_1 (x_{1i}; \beta) ; F_2 (x_{2i}; \beta))], \] where \(\gamma(\cdot, \cdot)\) is a sieve approximator of unknown true \(c(\cdot, \cdot)\).

Robust and non-redundant

• asy robust so long as \(\ln \gamma\) is consistent for true \(\ln c\).

• non-redundant so long as the score of \(\ln \gamma\) is not in the span of marginal scores.

• should improve on efficiency over QMLE up to semiparametric lower bound for efficiency (SPLB).

• never as precise as FMLE.

Sieve MLE

\[(\hat\beta, \hat\gamma) = \arg\max_{\beta, \gamma_N} \ln L(\beta; \gamma_N),\]

where \(\gamma_N\) is a sieve approximator of \(c_o\).

Approximation and denseness

• sieve is a set of approximating spaces.

• approximate \(\gamma \in \Gamma\) by parameters \(\gamma_N \in \Gamma_N\).

• \(\Gamma_N\) is dense in space of copula densities \(\Gamma\): \(\Gamma_N \rightarrow \Gamma\).

• sieve log-likelihood is maximised over \(\beta\) and \(\gamma_N\).

• once \(\dim{(\gamma_N)}\) is fixed, this is standard MLE over \(\{\beta, \gamma_N\}\).

Useful sieves: tensor products

Recall: \(c_o\) is a multivariate pdf with uniform marginals

\(M\)-variate tensor product of linear univariate sieves on \([0,1]\)

\[\Gamma_N=\left\{\gamma_N(\mathbf{u})= \exp\left\{\sum_{j=1}^{J_N}a_{1j} A_j(u_1)\cdot\ldots \cdot \sum_{j=1}^{J_N} a_{Mj}A_j(u_M) \right\}\right.,\\ \left.\mathbf{u}\in [0,1]^M, \int_{[0,1]^M} \gamma_N(\mathbf{u})d\mathbf{u}=1, \int_{[0,1]^{M-1}} \gamma_N(\mathbf{u})d\mathbf{u}^{-l}=1, \forall l \right\} \]

• \(\dim(a)=J_N^M\), all interactions of \(a_{ij}\)’s.
• basis functions \(A_j(u)\): power series, trigonometric polynomials, splines, or wavelets.
• proper copula conditions (integrability and marginal uniformity) not easy to satisfy with tensor product sieves.
• boundary problem – estimation bias at boundaries of support.
• boundary: copula densities unbounded at \(\{0,1\}\) and \(\{1,1\}\), e.g., Plackett.

Useful sieves: Bernstein

Bernstein-Kantorovich polynomial sieve

\[\Gamma_N = \left\{ \gamma_N(\mathbf{u})=J_N^M\sum_{j_1=0}^{J_N-1}\dots \sum_{j_M=0}^{J_{N}-1} \gamma(j_1, \ldots, j_M)\prod_{k=1}^M \underbrace{{J_N-1 \choose j_k} u_k^{j_k}(1-u_k)^{J_N-j_k-1}}_{\beta\text{-density}}\right\}, \] where \(\gamma(j_1, \ldots, j_M)\in[0,1]\) denotes \(J_N^M\) polynomial parameters.

Note:

• mixtures of \(\beta\)-densities \(\Rightarrow\) easy to impose the proper density condition and uniform marginals condition through \(\gamma\)’s.

• no boundary effects, uniform convergence on \([0,1]^M\).

• weights can be made adaptive and sparse – see later.

• infeasible if \(J_N^M>N \Rightarrow\) shrinkage, e.g., using L1-norm – see later.

Bivariate Bernstein copula

\[ c^B(u_1, u_2; \gamma_N) = \sum_{i=0}^{J_N-1} \sum_{j=0}^{J_N-1} \gamma_{ij} B_i(u_1) B_j(u_2), \] where \[ B_i(u) \equiv J_N {J_N-1 \choose i} u^i (1 - u)^{J_N-1-i} \qquad \gamma_N = \{\gamma_{ij}\}_{i,j=0, \ldots, J_N-1} \]

• can be viewed as histogram \(\gamma_{ij}\), over \(J_N \times J_N\) grid, smoothed with \(\beta\)-densities.

• \(\{\gamma_{ij}\}\) need to form a doubly stochastic matrix: \(\Gamma_N = \{\gamma_N \in \mathbb{R}_+^{J_N\times J_N}: \gamma_N \mathbf{1} = \frac{1}{J_N}\mathbf{1}, \gamma_N' \mathbf{1}= \frac{1}{J_N}\mathbf{1}\}\).

• this ensures uniform marginals and integration to 1.

• impose sparsity by requesting some \(\gamma_{ij} =0\) while preserving double stochasticity.

Sieve-based likelihood and moment conditions

\[ \ln L(\beta; \gamma_N) = \sum_{n=1}^N \left[\ln f_{1n}(\beta) + \ln f_{2n}(\beta) +\underbrace{\ln \sum_{i=0}^{J_N-1} \sum_{j=0}^{J_N-1} \gamma_{ij} B_i(F_{1n}(\beta)) B_j(F_{2n}(\beta))}_{\ln c_n^B(\beta; \gamma_N)}\right] \]

Given \(J_N\), this is standard MLE. Recall the 4 sets of moment conditions

\[\left.\begin{array}{c} \left. \begin{array}{c} (a)\quad\mathbb{E}\nabla_\beta \ln f_1(\beta_o) =0 \\ (b)\quad\mathbb{E}\nabla_\beta \ln f_2(\beta_o) =0 \end{array}\right\}\text{QMLE}\\ \left. \begin{array}{c} (c)\quad \mathbb{E}\nabla_\beta \ln c^B(\beta_o,\gamma_N) =0 \\ (d)\quad \mathbb{E}\nabla\gamma \ln c^B(\beta_o, \gamma_N) =0 \end{array}\right. \end{array}\right\}\text{SMLE} \]

Is it consistent?

Asymptotic normality and efficiency of SMLE

Standard MLE theory does not apply as \(J_N\rightarrow\infty\) in SMLE problem:

\[\hat \theta = \arg \max_{B \times \Gamma_N} \ln L(\beta,\gamma_N)\]

Let \(\theta=(\beta, \gamma)\). Consider functional \(\rho: \Theta \rightarrow R\); e.g. \(\rho(\theta)=\mathbb{E} \nabla \ln c\) or \(\rho(\theta)=\lambda'\beta\), \(\lambda \in \mathbb{R}^p\).

• This is a generalization of Delta methods, aka Slutsky Theorem, to functionals.

• Most efficient among semiparametric estimators using the same information

• If \(\sqrt N (\lambda'(\hat \beta-\beta_o)) \rightarrow N(0,\Omega)\) then \(\sqrt N(\hat \beta-\beta_o)\) is also asymptotically normal and efficient by the Cramer-Wold device

• (!) \(\sqrt{N}\) convergence, even if the nonparametric part of \(\hat{\theta}\) converges slower.

Efficiency bounds

Semiparametric asymptotic efficiency

• Model with nonparametric component is at least as hard as any one-dimensional submodel that satisfies semiparametric assumptions.

• Parameterise the semiparametric model: \(\theta(t)=\theta_o+t\nu\), where \(\nu \in\) tangent space.

• Estimating \(\theta_o\) is at least as hard as estimating \(t\) (“true” \(t=0\)).

• Semiparametric lower bound is the supremum over the traditional Cramer-Rao bounds (infimum over Fisher informations) for estimating \(t\) for all suitable parametric submodels.

Note:

\(I_{\theta(t)} = \mathbb{E}_{\theta_o} S_{\theta(t)} S_{\theta(t)}'\) \(\qquad\) \(S_{\theta(t)}=\frac{\partial l(\theta(t))}{\partial t}=\frac{\partial l(\theta)}{\partial \theta} [\nu]\).

SPLB \(= \sup_{\nu} I_{\theta(t)}^{-1} \ge I_F^{-1}\).

What is our SPLB?

Recall \(\rho(\theta)=\lambda'\beta\), semiparametric model \(\ln h(\theta_o) = \ln f_{1}(\beta_o) + \ln f_2(\beta_o)+\ln c^B(\beta_o,\gamma_o)\), parametric submodel \(\theta(t)=\theta_o+t\nu.\)

What is \(I_{\theta(t)}\) here?

Define directional derivative of sieve log-density in direction \(\nu = (\nu_\beta', \nu_\gamma')'\).

\[ S_{\theta(t)}=\dot{l}(\theta_o)[\nu] %\equiv \lim_{t\rightarrow 0} \left.\frac{\ln h (\theta_o+t\nu)- \ln h(\theta_o)}{t}\right|_{t=0} = \frac{\partial \ln h (\theta_o, x)}{\partial \theta'}[\nu]= \frac{\partial \ln h (\theta_o)}{\partial \beta'}\nu_\beta+\frac{\partial \ln h (\theta_o)}{\partial \gamma'}\nu_\gamma \]

Recall Fisher information for \(t\) is \(\mathbb{E}_{\theta_o} S_{\theta(t)} S_{\theta(t)}'\). So Fisher informations for estimating our submodels are \[I_{\theta_t}=\mathbb{E}_{\theta_o} \left[\dot{l}(\theta_o)[\cdot]\dot{l}(\theta_o)[\cdot] \right]\] and \[\text{SPLB} = \sup_{\nu \ne 0} I_{\theta_t}^{-1} \text{ over relevant submodels}.\]

SPLB and least favourable direction

Note:

Least favourable direction: close-form for Bernstein sieve

How much more efficiency?

Asy Relative Efficiency (ARE) of FMLE and SMLE vs QMLE: \[\frac{\mathbb{V}_\text{FMLE}}{\mathbb{V}_\text{QMLE}} \quad \& \quad \frac{\mathbb{V}_\text{SMLE}}{\mathbb{V}_\text{QMLE}}\]

Example: \(\beta\) = mean of exponential marginals

Natural sparsity

Many \(\gamma_{ij}\)’s in Bernstein approximator are close to zero. Copula densities are intrinsically sparse. Sparser for stronger dependence, e.g., as measured by Kendall’s \(\tau\):

Machine Learning

Can it help find the holy grail?

Sparse SMLE

Many ML methods exist (LASSO, RF, GBDT, DNN, etc).

Look at LASSO, i.e. L1-norm penalisation, on Bernstein parameters \(\gamma_{ij}\)’s.

How to estimate \(\beta_o\) consistently allowing for regularisation bias in estimating \(\gamma_o\)?

What won’t work

How to make ML estimators robust?

Want \(\hat\beta\) to be insensitive to deviations of \(\hat\gamma\).

Why does LASSO break?

Assume \(\dim(\gamma_N)<N\).

Recall the moments from FMLE:

\[\left.\begin{array}{c} \left. \begin{array}{c} (a)\quad\mathbb{E}\nabla_\beta \ln f_1(\beta_o) =0 \\ (b)\quad\mathbb{E}\nabla_\beta \ln f_2(\beta_o) =0 \end{array}\right\}\text{QMLE}\\ \left. \begin{array}{c} (c)\quad \mathbb{E}\!\nabla_\beta \ln k(\beta_0,\gamma_0^k)\ne 0 \\ (d)\quad \mathbb{E}\!\nabla_\gamma \ln k(\beta_0,\gamma_0^k)\ne 0 \end{array}\right. \end{array}\right\}\text{SMLE} \]

Penalisation invalides \((c)\) and \((d)\) even if they were correct. This causes inconsistency of \(\hat\beta\) There is a pseudo-true value that sets relevant F.O.C. equal to zero.

\[(*)\quad\left\{\begin{matrix} \mathbb{E}\nabla_\beta \ln c^B(\beta_o^*,\gamma_N^*) =0 \\ \mathbb{E}\nabla\gamma \ln c^B(\beta_o^*, \gamma_N^*) =0 \end{matrix} \right. \]

LASSO bias carries over to \(\hat\beta\). When does estimation of \({\gamma}\) not affect estimation of \(\beta\)?

When does estimation of \({\gamma}\) not affect estimation of \(\beta\)?

Turns out our earlier results apply:

Whenever it is irrelevant for the estimation of \(\beta\) whether we know \((d)\) and the true \(\gamma_o\).

Let \(g_1=s_\beta = (a)+(b)+(c)\) and \(g_2=s_\gamma=(d)\).

Notation: \(\quad \mathbb{D}_1=\left[\begin{array}{c}\mathbb{E}\nabla_\theta g_1 \\ \mathbb{E}\nabla_\theta g_2\end{array}\right] = \left[\begin{array}{c}\mathbb{D}_1 \\ \mathbb{D}_2\end{array}\right]\qquad \mathbb{C}=\left[\begin{array}{cc}\mathbb{E}g_1 g_1'&\mathbb{E}g_1 g_2' \\ \mathbb{E}g_2 g_1'&\mathbb{E}g_2 g_2'\end{array}\right] =\mathbb{V}\left[\begin{array}{c} g_{1}\\ g_{2}\end{array}\right] = \left[\begin{array}{cc} \mathbb{C}_{11}& \mathbb{C}_{12}\\ \mathbb{C}_{21}&\mathbb{C}_{22}\end{array}\right]\)

M/P-Redundancy \[ \Leftrightarrow \begin{array}{cl} \mathbb{C}_{12}=0 &\text{Moment redundancy} \\ \mathbb{D}_{12}=\mathbb{E}\nabla_\gamma g_{1}(\beta_o, \gamma_N)=0 & \text{Parameter redundancy} \end{array}\]

Look for a valid moment function \(g^*_{1}(\beta, \gamma)\) uncorrelated with \(g_{2}\) such that \[\mathbb{D}_{12}=\mathbb{E}\nabla_\gamma g_{1}^*(\beta_o, \gamma_N)=0.\]

Obvious answer: use error in projection

\[ \boxed{ g^*_1(\beta, \gamma) = g_{1}(\beta_o, \gamma_N) - \mathbb{C}_{12}\mathbb{C}_{22}^{-1}g_{2}(\beta_o, \gamma_N) } \]

Neyman orthogonality

Now let \(d=\dim(\gamma)=\infty\) [\(\gamma\) is a function]

In context of asy optimal testing: when errors in \(\hat{\gamma}\) do not carry over into \(\hat\beta\)

A moment is Neyman orthogonal if it is locally insensitive, to first order, to perturbations of the nuisance parameter.

Formally, for every admissible direction \(\gamma\) in the nuisance tangent set,

\[\textbf{Neyman Orthogonality } \Leftrightarrow\left.\begin{array}{cl} \mathbb{D}_{12}[\gamma-\gamma_o]=0 & \text{ Gateaux derivative} \end{array}\right.\]

Look for a valid moment function \(g^*_{1}(\beta, \gamma)\) such that

\[\mathbb{D}_{12}[\gamma-\gamma_o] = \partial_\gamma \mathbb{E}g_{1}^*(\beta_0, \gamma_o)[\gamma-\gamma_o] = \lim_{\tau\rightarrow 0} \frac{\mathbb{E}g_{1}^*(\beta_o, \gamma_o+\tau(\gamma-\gamma_o))-\mathbb{E}g_{1}^*(\beta_o, \gamma_o)}{\tau}=0\]

Turns out this is the orthogonal score we found when establishing SPLB.

Close-form for Bernstein sieve:

\[s_{\beta}^* = \underbrace{s_{\beta}}_{\text{raw score}} - \underbrace{\frac{b(u)'} {c_B(u; \gamma)}\left[\mathbb{E}\frac{b(u)b(u)'}{c_B(u;\gamma)}\right]^{-1} \mathbb{E}s_\beta b(u)}_{\text{projection onto nuisance space}}.\]

Use cross-fitting, to estimate \(\gamma\) and projection on the auxiliary sample, while this moment equation is evaluated on the held-out sample.

Profile likelihood

Equivalent – see paper \[ \alpha^*_\beta = \arg \min_{\alpha \in R^{J_N^2}} - \frac{1}{N}\sum_{n=1}^N \sum_{i, j} \gamma(\alpha_{ij}^2) B_i(F_{1n}(\beta)) B_j(F_{2n}(\beta)) + \lambda \sum_{i, j} |\alpha_{ij}|, \]

where we use reparameterisation \(\gamma_{ij} = \gamma(\alpha_{ij}^2)\) via Sinkhorn algorithm to impose doubly stochastic \(\gamma_{ij}\) while keeping sparsity intact.

Double machine learning SMLE algorithm

• Obtain a consistent estimate of \(\beta\) using QMLE, denote it \(\hat{\beta}_a\).

• Divide data into \(K\) folds. For each fold \(k=1\) to \(K\), do the following:

– Let data in fold \(k\) be the main sample \(D_{m}(k)\). Let the rest of data be the auxilary sample \(D_{a}(k)\).

– Obtain the solution to profile likelihood problem using \(\hat{\beta}_a\) as initial values in three steps:

Step 1. use \(D_{a}(k)\) to obtain estimates of penalised \(\gamma_{ij}\).

Step 2. use \(D_{a}(k)\) to obtain estimates of \(\gamma_{ij}\) using only the non-zero \(\gamma_{ij}\) picked in Step 1.

Step 3. use \(D_{main}(k)\) to obtain \(\hat{\beta}(k)\).

• Estimate \(\beta\) as the average over folds: \[ \hat{\beta} = \frac{1}{K} \sum_{k=1}^K \hat{\beta}(k) \]

Asymptotics for DML-SMLE

E.g., Naive SMLE and DML-SMLE for \(\mu_o=1\), Plackett copula, near lower Frechet bound. \(N=1000\), based on 1000 reps:

Conclusions and promising directions

• robust and efficiency-improving copulas are a Holy Grain of dependence modeling.

• parametric robust and efficiency-improving copulas are rare.

• parametric copulas are redundant if their scores are in a linear span of marginal scores.

• robust copulas are often robust because they are redundant.

• sieves offer an attractive nonparametric alternative in cases of strong dependence.

• Bernstein sieve offers an close-form for SPLB and penalised robust SMLE.

• Neyman orthogonality can be viewed in terms of moment redundancy.

• double machine learning helps achieve robustness and improve efficiency over naive penalised SMLE.

• many directions for future work including specific results for non-LASSO, non-Bernstein maching learning methods known to handle high dimensions and sparsity effectively, e.g., DNN, GRF, MIP