(^ie1ddlmZddlZddlmZddlmZddlmZddlZddl Z ddl Z ddl Z ddl m Z ddlZddlZddlmZddlmZd Zd ZGd d eZGd dej.ZGddeZGddeZed#dZedddGddeZeddddGddeZedddd Gd!d"eZy)$) annotationsN) Generator)Iterator)contextmanager)Any)deprecated_class)BaseJournalBackendz.lockz.renamec*eZdZdZdddZddZd dZy) JournalFileBackendaFile storage class for Journal log backend. Compared to SQLite3, the benefit of this backend is that it is more suitable for environments where the file system does not support ``fcntl()`` file locking. For example, as written in the `SQLite3 FAQ `__, SQLite3 might not work on NFS (Network File System) since ``fcntl()`` file locking is broken on many NFS implementations. In such scenarios, this backend provides several workarounds for locking files. For more details, refer to the `Medium blog post`_. .. _Medium blog post: https://medium.com/optuna/distributed-optimization-via-nfs -using-optunas-new-operation-based-logging-storage-9815f9c3f932 It's important to note that, similar to SQLite3, this class doesn't support a high level of write concurrency, as outlined in the `SQLAlchemy documentation`_. However, in typical situations where the objective function is computationally expensive, Optuna users don't need to be concerned about this limitation. The reason being, the write operations are not the bottleneck as long as the objective function doesn't invoke :meth:`~optuna.trial.Trial.report` and :meth:`~optuna.trial.Trial.set_user_attr` excessively. .. _SQLAlchemy documentation: https://docs.sqlalchemy.org/en/20/dialects/sqlite.html #database-locking-behavior-concurrency Args: file_path: Path of file to persist the log to. lock_obj: Lock object for process exclusivity. An instance of :class:`~optuna.storages.journal.JournalFileSymlinkLock` and :class:`~optuna.storages.journal.JournalFileOpenLock` can be passed. Nc||_|xst|j|_tjj |js$t |jdjddi|_y)Nabr) _file_pathJournalFileSymlinkLock_lockospathexistsopenclose_log_number_offset)self file_pathlock_objs ^/var/www/html/hubwallet-dev/venv/lib/python3.12/site-packages/optuna/storages/journal/_file.py__init__zJournalFileBackend.__init__8sV(H!7!H ww~~doo. $ ' - - /34a&c#Kt|jd5}tj|jj}d}||j vr2|j |j ||}||j |z}d}t||D]\}}t|}||z}|dkrn|||dz|j vr"|j ||z|j |dz<||krZ|jdstd}|j |dz= tj|dddy#tj$r} | }|j |dz=Yd} ~ d} ~ wwxYw#1swYyxYww)Nrbr)start zInvalid log format.)rrrstatst_sizerseek enumeratelenendswith ValueErrorjsonloadsJSONDecodeError) rlog_number_fromfremaining_log_sizelog_number_startlast_decode_error log_numberlinebyte_lenerrs r read_logszJournalFileBackend.read_logs?s $//4 (" @A"$!9!A!A  $"9"99t..?@#2 "d&=&=o&NN" $ $-a7G$H @ Dt9"h."%)$0++>)@)@@// ;hF++JN;/}}U+(23H(I%// Q?@**T**) @" @" @@++@(+%// Q?@A" @" @sGE<DE0D>3E05 E<>E-E(#E0(E--E00E95E<c t|j5dj|Dcgc]}tj|dc}dz}t |j d5}|j|jd|jtj|jddddddycc}w#1swYxYw#1swYyxYw)N ),:) separatorsr zutf-8) get_lock_filerjoinr)dumpsrrwriteencodeflushrfsyncfileno)rlogslog what_to_writer-s r append_logszJournalFileBackend.append_logsds 4:: & % TRc4::cjARSVZZ doot, % ,,W56 $ %  % %R % %  % %s5CC  C$AC8C CC CC#N)rstrrzBaseJournalFileLock | NonereturnNone)r,intrIz%Generator[dict[str, Any], None, None])rCzlist[dict[str, Any]]rIrJ)__name__ __module__ __qualname____doc__rr5rFrrr r s@9#@J%rr cXeZdZejddZejddZy)BaseJournalFileLockctrGNotImplementedErrorrs racquirezBaseJournalFileLock.acquirep!!rctrGrTrVs rreleasezBaseJournalFileLock.releasetrXrNrIboolrIrJ)rLrMrNabcabstractmethodrWrZrPrrrRrRos4"" ""rrRc*eZdZdZdddZddZd dZy) raLock class for synchronizing processes for NFSv2 or later. On acquiring the lock, link system call is called to create an exclusive file. The file is deleted when the lock is released. In NFS environments prior to NFSv3, use this instead of :class:`~optuna.storages.journal.JournalFileOpenLock`. Args: filepath: The path of the file whose race condition must be protected. grace_period: Grace period before an existing lock is forcibly released. c||_|tz|_|*|dkr td|dkrt j d||_yNrz9The value of `grace_period` should be a positive integer.z0The value of `grace_period` might be too small. )_lock_target_fileLOCK_FILE_SUFFIX _lock_filer(warningswarn grace_periodrfilepathris rrzJournalFileSymlinkLock.__init__sN!)"%55  #q  !\]]a PQ(rcd}tj}d} tj|j|j y#t $r}|jtjk(r|j tj|j j}n#t $rYYd}~wxYw||k7r|}tj}tj|z |jkDr=tjd |jd}n#t$rYYd}~wxYwtj |t#|dzd}Yd}~D|d}~wt$$r|jwxYw)zAcquire a lock in a blocking way by creating a symbolic link of a file. Returns: :obj:`True` if it succeeded in creating a symbolic link of ``self._lock_target_file``. MbP?NTfThe existing lock file has not been released for an extended period. Forcibly releasing the lock file.r )time monotonicrsymlinkrdrfOSErrorerrnoEEXISTrir"st_mtimergrhrZ RuntimeErrorsleepmin BaseException)r sleep_secslast_update_monotonic_timemtimer4 current_mtimes rrWzJournalFileSymlinkLock.acquires?  %)^^%5"  4114??C 99 ,((4%,.GGDOO,D,M,MM&%$%(E1$1E9=9I6>>+.HH4K\K\\$MM!\) $ -2 #/) ()JJz*!$Z!^Q!7J     sf*A E=)E:)B$#E$ B4-E3B44AE DE D0(E/D00'EEE=c>|jttjztz} t j |j|t j|y#t$r tdt$rt j|wxYw)z-Release a lock by removing the symbolic link.Error: did not possess lockN rfrHuuiduuid4RENAME_FILE_SUFFIXrrenameunlinkrsrwrzrlock_rename_files rrZzJournalFileSymlinkLock.release| ??S->>ASS  IIdoo'7 8 II& ' ><= =  II& '   5A))3BNrkrHriz int | NonerIrJr[r]rLrMrNrOrrWrZrPrrrrys ))V rrc*eZdZdZdddZddZd dZy) JournalFileOpenLockaPLock class for synchronizing processes for NFSv3 or later. On acquiring the lock, open system call is called with the O_EXCL option to create an exclusive file. The file is deleted when the lock is released. This class is only supported when using NFSv3 or later on kernel 2.6 or later. In prior NFS environments, use :class:`~optuna.storages.journal.JournalFileSymlinkLock`. Args: filepath: The path of the file whose race condition must be protected. grace_period: Grace period before an existing lock is forcibly released. c|tz|_|*|dkr td|dkrtjd||_yrb)rerfr(rgrhrirjs rrzJournalFileOpenLock.__init__sF"%55  #q  !\]]a PQ(rcvd}tj}d} tjtjztj z}tj tj|j|y#t$r}|jtjk(r|j tj|jj}n#t$rYYd}~wxYw||k7r|}tj}tj|z |jkDr=tj d |j#d}n#t$$rYYd}~QwxYwtj&|t)|dzd}Yd}~|d}~wt*$r|j#wxYw)zAcquire a lock in a blocking way by creating a lock file. Returns: :obj:`True` if it succeeded in creating a ``self._lock_file``. rmNTrnror )rprqrO_CREATO_EXCLO_WRONLYrrrfrsrtrurir"rvrgrhrZrwrxryrz)rr{r|r} open_flagsr4r~s rrWzJournalFileOpenLock.acquires] %)^^%5" ZZ"))3bkkA *=> 99 ,((4%,.GGDOO,D,M,MM&%$%(E1$1E9=9I6>>+.HH4K\K\\$MM!\) $ -2 #/) ()JJz*!$Z!^Q!7J     sgA%B F8 )F5)CF C/(F.C//AFEF E+#F*E++'FFF8c>|jttjztz} t j |j|t j|y#t$r tdt$rt j|wxYw)z,Release a lock by removing the created file.rNrrs rrZzJournalFileOpenLock.releaserrNrrr[r]rrPrrrrs )+Z rrc#K|j d|jy#|jwxYwwrG)rWrZ)rs rr;r;s2  s>)>;>z4.0.0z6.0.0zAUse :class:`~optuna.storages.journal.JournalFileBackend` instead.)textc eZdZy)JournalFileStorageNrLrMrNrPrrrr&s rrz=The import path :class:`~optuna.storages.JournalFileOpenLock`zBUse :class:`~optuna.storages.journal.JournalFileOpenLock` instead.)deprecated_versionremoved_versionnamerc eZdZy)DeprecatedJournalFileOpenLockNrrPrrrr- rrz@The import path :class:`~optuna.storages.JournalFileSymlinkLock`zEUse :class:`~optuna.storages.journal.JournalFileSymlinkLock` instead.c eZdZy) DeprecatedJournalFileSymlinkLockNrrPrrrr7rrr)rrRrIzIterator[None]) __future__rr^collections.abcrr contextlibrrtr)rrptypingrrrgoptuna._deprecatedroptuna.storages.journal._baser rerr ABCrRrrr;rrrrPrrrs" %$%  /<U%+U%p"#''"N0NbP-Pf W^ +   H M   $7    K P   '=   r