(^i-ddlmZddlZddlZddlZddlZddlmZejdejzZ eje Z ejdejz ZejdZddZddZej$ddd Zdd Zej$ddd Zdd Zdd ZddZddZddZ d ddZ d ddZy)) annotationsNerfc.tj||SN)np logaddexplog_plog_qs `/var/www/html/hubwallet-dev/venv/lib/python3.12/site-packages/optuna/samplers/_tpe/_truncnorm.py_log_sumr3s <<u %%c`|tjtj||z  zSr )r log1pexpr s r _log_diffr7s& 288RVVEEM223 33ric|dz }|dkrdtj| z}|S|dkrddtj|zz}|Sddtj|zz }|S)N;f?g;f?g;f??)matherfcr)axys r _ndtr_singler;sr F A; $))QB-  H Z # # # H # ! $ $ Hrc*ddt|dz zzS)Nrrrrs r_ndtrr"Is s1v:& &&rcF|dkDr t|  S|dkDrtjt|Sd|dzztj| z dtjdtjzzz }d}d}d}d}d|dzz }d}d }t ||z t j jkDrO|dz }|}| }||z}|d|zdz z}|||z|zz }t ||z t j jkDrO|tj|zS) Nirrgrr)rrlogpiabssys float_infoepsilon) rlog_LHS last_totalright_hand_side numerator denom_factor denom_conssignis r_log_ndtr_singler5Ns*1uaR   3wxx Q((QTkDHHaRL(3!dgg+1F+FFGJOILQTJ D A j?* +cnn.D.D D Q$ u " QUQY 4)+l:: j?* +cnn.D.D D TXXo. ..rcjtjtdd|jtS)Nr&)r frompyfuncr5astypefloatr!s r _log_ndtrr:is( 02==)1a 0 3 : :5 AArc"|dz dz tz S)Nrg@)_norm_pdf_logC)rs r _norm_logpdfr=ms T7S=> ))rc |dk}|dkD}||z}dd d fd }dd}tj|tjtj}||x}jr |||||<||x} jr|| ||||<||x} jr|| ||||<tj |S)z3Log of Gaussian probability mass within an intervalrc>tt|t|Sr )rr:rbs rmass_case_leftz'_log_gauss_mass..mass_case_leftzs1y|44rc| | Sr )rrArBs rmass_case_rightz(_log_gauss_mass..mass_case_right}sqb1"%%rcZtjt| t| z Sr )r rr"r@s rmass_case_centralz*_log_gauss_mass..mass_case_centrals$xxq E1"I-..r) fill_valuedtyper np.ndarrayrArKreturnrK)r full_likenan complex128sizereal) rrA case_left case_right case_centralrErGouta_lefta_right a_centralrBs @r_log_gauss_massrYqs QIQJ+,L5& / ,,qRVV2== ACI,$$') =IZ= &&)'1Z=AJ|_$ **-i<IL 773<rcH|dkD}|j}tjtj|| ||<tj|}tj |dkx}dj r&tjd||tzz ||<tj |dk\x}dj r7t tjtj|| z||<tdD]}t|}d|dzztz }||z tj||z z} || z}tjtj| dtj|zksn||xxd zcc<|S) a\ Use the Newton method to efficiently find the root. `ndtri_exp(y)` returns `x` such that `y = log_ndtr(x)`, meaning that the Newton method is supposed to find the root of `f(x) := log_ndtr(x) - y = 0`. Since `df/dx = d log_ndtr(x)/dx = (ndtr(x))'/ndtr(x) = norm_pdf(x)/ndtr(x)`, the Newton update is x[n + 1] := x[n] - (log_ndtr(x) - y) * ndtr(x) / norm_pdf(x). As an initial guess, we use the Gaussian tail asymptotic approximation: 1 - ndtr(x) \simeq norm_pdf(x) / x --> log(norm_pdf(x) / x) = -1/2 * x**2 - 1/2 * log(2pi) - log(x) First recall that y is a non-positive value and y = log_ndtr(inf) = 0 and y = log_ndtr(-inf) = -inf. If abs(y) is very small, we first derive -x such that z = log_ndtr(-x) and then flip the sign. Please note that the following holds: z = log_ndtr(-x) --> z = log(1 - ndtr(x)) = log(1 - exp(y)) = log(-expm1(y)). Recall that as long as ndtr(x) = exp(y) > 0.5 --> y > -log(2) = -0.693..., x becomes positive. ndtr(x) = exp(y) \simeq 1 + y --> -y \simeq 1 - ndtr(x). From this, we can calculate: log(1 - ndtr(x)) \simeq log(-y) \simeq -1/2 * x**2 - 1/2 * log(2pi) - log(x). Because x**2 >> log(x), we can ignore the second and third terms, leading to: log(-y) \simeq -1/2 * x**2 --> x \simeq sqrt(-2 log(-y)). where we take the positive `x` as abs(y) becomes very small only if x >> 0. The second order approximation version is sqrt(-2 log(-y) - log(-2 log(-y))). If abs(y) is very large, we use log_ndtr(x) \simeq -1/2 * x**2 - 1/2 * log(2pi) - log(x). To solve this equation analytically, we ignore the log term, yielding: log_ndtr(x) = y \simeq -1/2 * x**2 - 1/2 * log(2pi) --> y + 1/2 * log(2pi) = -1/2 * x**2 --> x**2 = -2 * (y + 1/2 * log(2pi)) --> x = sqrt(-2 * (y + 1/2 * log(2pi)) For the moderate y, we use Eq. (13), i.e., standard logistic CDF, in the following paper: - `Approximating the Cumulative Distribution Function of the Normal Distribution `__ The standard logistic CDF approximates ndtr(x) with: exp(y) = ndtr(x) \simeq 1 / (1 + exp(-pi * x / sqrt(3))) --> exp(-y) \simeq 1 + exp(-pi * x / sqrt(3)) --> log(exp(-y) - 1) \simeq -pi * x / sqrt(3) --> x \simeq -sqrt(3) / pi * log(exp(-y) - 1). g{Gzrgdr%rg:0yE>)copyr r'expm1 empty_likenonzerorPsqrtr<_ndtri_exp_approx_Cranger:rallr)) rflippedzr small_inds moderate_inds_ log_ndtr_xlog_norm_pdf_xdxs r _ndtri_exprnsm^%iG A!G*--.AgJ aAjjR(( !,11:)G!HII* AG,, a055//"&&1]CSBS9T2UU- 3Z q\ 1~51nzN'B C C R 66"&&*tbffQi// 0  gJ"J Hrctj|||\}}}tj|||\}}}|dk}|}t||}dd}dd}tj|}||x} j r|| ||||||||<||x} j r|| ||||||||<tj ||k(|dk(|dk(gtj||g|S)a= Compute the percent point function (inverse of cdf) at q of the given truncated Gaussian. Namely, this function returns the value `c` such that: q = \int_{a}^{c} f(x) dx where `f(x)` is the probability density function of the truncated normal distribution with the lower limit `a` and the upper limit `b`. More precisely, this function returns `c` such that: ndtr(c) = ndtr(a) + q * (ndtr(b) - ndtr(a)) for the case where `a < 0`, i.e., `case_left`. For `case_right`, we flip the sign for the better numerical stability. rcntt|tj||z}t |Sr )rr:r r'rnqrrAlog_mass log_Phi_xs rppf_leftzppf..ppf_lefts*Yq\266!9x+?@ )$$rcttt| tj| |z}t | Sr )rr:r rrnrqs r ppf_rightzppf..ppf_rights3Yr]BHHaRL8,CD 9%%%rr&default) rrrKrrKrArKrsrKrLrK) r atleast_1dbroadcast_arraysrYr`rPselectrrN) rrrrArRrSrsrurwrUq_leftq_rights rppfrsmmAq!$GAq!!!!Q*GAq!AIJq!$H%& -- CI,$$!&!I,) hyFYZIZ= &&#GQz]AjM8T^K_`J 99a1fa1fa1f-!Q/? MMrc|xstjj}tj||||j}|j dd|}t ||||z|zS)z This function generates random variates from a truncated normal distribution defined between `a` and `b` with the mean of `loc` and the standard deviation of `scale`. rr&)lowhighrP)r random RandomState broadcastshapeuniformr)rrAlocscale random_staterP quantiless rrvsr se :299#8#8#:L <<1c5 ) / /D$$$>I y!Q % '# --rcd||z |z }tj|||\}}}t|t||z tj|z }tj |||\}}}tj ||k(||k||kDzgtjtj g|S)Nrx) r rzr=rYr'r{r|rNinf)rrrArrrUs rlogpdfrs SEAmmAq!$GAq! q/OAq1 1BFF5M AC!!!Q*GAq! 99a1fq1uQ/0266BFF72CS QQr)r rKrrKrLrK)rr9rLr9)rrKrLrK)rrKrLrKrJ)rrKrLrK)rrrKrnp.ndarray | floatrArrLrK)rr&N) rrKrArKrrrrrznp.random.RandomState | NonerLrK)rr&) rrKrrrArrrrrrLrK) __future__r functoolsrr*numpyr optuna.samplers._tpe._erfrrbr( _norm_pdf_Cr'r<rc_log_2rr lru_cacherr"r5r:r=rYrnrrrrDrrrspD# )diiDGG $ +&diilTWW, !&4T    ' T//4B*$NE P&NX !15 ... .  . / .  .* ! R R R R  R  R  Rr