U kf@sdZddlmZddlmZddlmZddlmZddlZddlm Z dd lm Z dd lm Z dd lm Z dd lmZd d lm Zd dlmZd dlmZd dlmZd dlmZd dlmZd dlmZd dlmZdDddZddZddZddZddZ d d!Z!dddd"d#d$d%Z"ddd"d&d'd(Z#d)d*Z$d+d,Z%d-dd-d.d/d0Z&d-ddd1d2d3Z'd4d5Z(d6d7Z)d8d9Z*d:d;Z+dd?Z-d@dAZ.dBdCZ/dS)Ezprivate module containing functions used to emit INSERT, UPDATE and DELETE statements on behalf of a :class:`_orm.Mapper` and its descending mappers. The functions here are called only by the unit of work functions in unitofwork.py. ) annotations)chain)groupby) zip_longestN) attributes)exc)loading)sync) state_str)future)sql)util)cursor) operators)BooleanClauseList)LABEL_STYLE_TABLENAME_PLUS_COLFcCs|s0|js0t||D]}t||g|ddqdSg}g}t|||D]B\}}}} } } } | s^| rt||||| | fqD||||| fqD|jD]J\} }| |jkrqt| |}t || |}t |||| |t |||| |qt ||t dd|Ddd|DdS)a;Issue ``INSERT`` and/or ``UPDATE`` statements for a list of objects. This is called within the context of a UOWTransaction during a flush operation, given a list of states to be flushed. The base mapper in an inheritance hierarchy handles the inserts/ updates for all descendant mappers. T)singleNcss$|]\}}}}||||dfVqdS)FN).0state state_dictmapper connectionrrJ/opt/hc_python/lib64/python3.8/site-packages/sqlalchemy/orm/persistence.py is zsave_obj..css&|]\}}}}}||||dfVqdS)TNr)rrrrrupdate_version_idrrrrms)batch _sort_statessave_obj_organize_states_for_saveappend_sorted_tablesitems _pks_by_table_collect_insert_commands_collect_update_commands_emit_update_statements_emit_insert_statements _finalize_insert_update_commandsr) base_mapperstatesuowtransactionrrstates_to_updatestates_to_insertdict_rr has_identity row_switchrtableinsertupdaterrrr (sn     r cshtt|||}|jD]H\jkr.qfdd|D}t||||}t|||qdS)z\Issue UPDATE statements on behalf of a relationship() which specifies post_update. c 3sH|]@\}}}}|jkr||||jdk r:||jndfVqdSN)r%version_id_col#_get_committed_state_attr_by_column)rrrZ sub_mapperrrr3rrrs  zpost_update..N)list _organize_states_for_post_updater#r$r%_collect_post_update_commands_emit_post_update_statements)r+r,r-post_update_colsr.r5rr9r post_update{s.   r?c Cstt|||}|j}tt|D]F}||}||jkr@q&n|jrN|jrNq&t||||}t |||||q&|D]\}} }} } |j || |qrdS)zIssue ``DELETE`` statements for a list of objects. This is called within the context of a UOWTransaction during a flush operation. N) r:_organize_states_for_deleter#reversedkeysr%ZinheritsZpassive_deletes_collect_delete_commands_emit_delete_statementsdispatchZ after_delete) r+r,r-states_to_deleteZtable_to_mapperr3rdeleterrrrrrr delete_objs>   rHc csJt|||D]6\}}}}t|j}|jp2||}d} } |sR|j|||n|j||||jrv|||||s||jj kr|jj |} t | } | | s| | stdt||t| fn&|d|t|t| || | } |s| r0|jdk r0|| r| n|| r&| jn||j} |||||| | fVq dS)aTMake an initial pass across a set of states for INSERT or UPDATE. This includes splitting out into distinct lists for each, calling before_insert/before_update, obtaining key information for each state including its dictionary, mapper, the connection to use for the execution per state, and the identity flag. NzJNew instance %s with identity key %s conflicts with persistent instance %szPdetected row switch for identity %s. will update %s, remove %s from transaction)_connections_for_statesboolkey_identity_key_from_staterEZ before_insertZ before_updateZ_validate_polymorphic_identitysessionZ identity_maprZinstance_stateZwas_already_deleted is_deletedrwarnr Z _log_debugZremove_state_actionsr7r8dict) r+r,r-rr0rrr1Z instance_keyr2rinstanceexistingrrrr!s`         r!cCs t|||S)zMake an initial pass across a set of states for UPDATE corresponding to post_update. This includes obtaining key information for each state including its dictionary, mapper, the connection to use for the execution per state. )rI)r+r,r-rrrr;#s r;ccs^t|||D]L\}}}}|j||||jdk rD||||j}nd}|||||fVq dS)zMake an initial pass across a set of states for DELETE. This includes calling out before_delete and obtaining key information for each state including its dictionary, mapper, the connection to use for the execution per state. N)rIrEZ before_deleter7r8)r+r,r-rr0rrrrrrr@/s r@r)bulkreturn_defaults render_nullsinclude_bulk_keysc #s|D]\}}}||jkrqi} i} |j|} |j|} t| D]l} | }| | }|dkrt|| krt|stqHqH|st|dst|tjrt|dr| n|| |<qH|| |j <qH|s|j | |  dd| DD] }d| |<q|r|r,|j || }|j|j|r&|j|| }nd}nd}}|jdk rp|jdk rp|j|j|krp|d| |jj <|r|jr| |j|j|r| fdd|D|| ||| ||fVqdS) zSIdentify sets of values to use in INSERT statements for a list of states. N__clause_element__cSsg|] }|jqSrrKrcrrr ysz,_collect_insert_commands..TFc3s|]}||fVqdSr6rrkrrrrsz+_collect_insert_commands..)r%_propkey_to_colZ_insert_cols_evaluating_noneset intersectionhasattr isinstancer ClauseElementrWrKZ_insert_cols_as_none differenceZ_pk_keys_by_tableissubsetr+_prefer_eager_defaultsdialectZ_server_default_col_keysversion_id_generatorr7_cols_by_tableZ_set_polymorphic_identity setdefaultZ_polymorphic_attr_keyZpolymorphic_identityr5)r3r/rSrTrUrVrrrparams value_paramspropkey_to_colZ eval_nonepropkeyvaluecolZcolkey has_all_pkshas_all_defaultsrr^rr&Fs            r&)rSuse_orm_update_stmtrVc #s|D]\}}}} ||jkr q|j|} |dk r:|j} ni} |j||r|fddt|j|D} d} ni} t|jD]n}|}|}t|dst |t j rt|dr| n|| |<q|j |j||j|dk r|| |j<q|jjdkr |j|| } nd} | dk r|j|j|kr|s| s| s|jD].}|j |jj|tj}|jr\qq\q|j}| o| }| | |j<|s|j| kr|jdk r|| }|| |j<n|jdkr |r | | |j<n| s | s qd}d}|r|fddt|j|Dtj rHt!j"dd #fd d | Dd d dni| D]}|j|j}|j |j|tj$}|jr|j%rd||f|jkrd}|jd|j<| &|jdn|j%d|j<|| kr"d}n|j'd|j<|jdkrt()d||fq|rf| *fdd |D| sr| r| *|| ||| | |fVq|r|j+|D]"\}}t,-|||||||j.qqdS)atIdentify sets of values to use in UPDATE statements for a list of states. This function works intricately with the history system to determine exactly what values should be updated as well as how the row should be matched within an UPDATE statement. Includes some tricky scenarios where the primary key of an object might have been changed. Ncsi|]}|j|qSrrXrrornrrr sz,_collect_update_commands..TrWFcsi|]}|j|qSr)_labelgetrurvrrrw)sz,No primary key value supplied for column(s) z, c3s$|]}|jdkrt|VqdSr6)rxstrrY) pk_paramsrrr2sz+_collect_update_commands..zY; per-row ORM Bulk UPDATE by Primary Key requires that records contain primary key valuesZbupq)codeZ pk_cascadedrzCCan't update table %s using NULL for primary key value on column %sc3s|]}||fVqdSr6rr\r^rrr\s)/r%Z_valuesr_r`rareZ_pk_attr_keys_by_tablecommitted_staterbrcrrdrWmanagerimplZis_equalrKr+eager_defaultsZ!_server_onupdate_default_col_keysrfr7rj_columntopropertyvalues get_historyrPASSIVE_NO_INITIALIZEaddedrxrirZNONE_SETsa_excInvalidRequestErrorjoin PASSIVE_OFFZdeletedpopZ unchangedorm_exc FlushErrorr5_table_to_equatedr populatepassive_updates)r-r3r.rSrtrVrrrrpksrmrlrsrorprqprophistoryZ no_paramsvalrrZexpect_pk_cascadedm equated_pairsr)r{rnrrr's*                      r'ccs,|D] \}}}}} |j|} i} d} |j|D]|} | | krX|j||| tjd| | j<q0| |ksj| jdk r0|j| }|j|j j ||tj }|j r0|j d}|| | j <d} q0| r| dk r|j|j|kr|j} | | | j<t|j r| j | kr|jdk r|| }|| | j <||||| fVqdS)zrIdentify sets of values to use in UPDATE statements for a list of states within a post_update operation. F)ZpassiveNrT)r%rjZ_get_state_attr_by_columnrrrxZonupdaterr~rKrrrrr7rJri)r+r-r3r.r>rrrrrrrlZhasdatarqrrrprrrrr<|sZ       r<c cs|D]\}}}}}||jkrqi} |j|D]6} |||| | | j<} | dkr,td|| fq,|dk r|j|j|kr|| |jj<| |fVqdS)zSIdentify values to use in DELETE statements for a list of states to be deleted.NzHCan't delete from table %s using NULL for primary key value on column %s)r%r8rKrrr7rj) r+r-r3rFrrrrrrlrqrprrrrCs:    rCT) bookkeepingrtenable_check_rowcountc sjdk ojjkd|ji}dfdd } |dk rH| |} n|df| } t|ddD]\\} } } }}}d}t|}| }|dk r|d }d }|s|jj}d }|r|sj j d krj r| j j r|jj}d }jr |j}d }| j j}|o | j j}| o. }| r|D]j\}}}} }}}| j||||d }|rt|||||jjd|d |j ||j7}|o|}q:n |s|o|}|D]Z\}}}} }}}| j|||d }|r t|||||jjd|d |j ||j7}qnd d|D}|oF|pF|oFt|dk}| j|||d }||j7}|D]N\}}}} }}}|rft|||||jjd|d |jjs|jnd qf|r|t|krtdjt||fqfrftd|j j qfdS)z`Emit UPDATE statements corresponding to value lists collected by _collect_update_commands().Ncompiled_cachecsttj}jD] }||tj|j|j dkqr^|j tjj jj j dk|dk rr| |}n |}|SN)type_) r_construct_rawrand_r%_append_inplacer bindparamrxtyper7wherer5)Z existing_stmtclausesrqstmtrneeds_version_idr3rr update_stmts$  z,_emit_update_statements..update_stmtr5cSs*|dt|dt|d|d|dfSNr r`rJrecrrrs   z)_emit_update_statements..r)Z_emit_update_tableZ_emit_update_mapperFTexecution_optionscSsg|] }|dqSr rrrrrrr[sz+_emit_update_statements..rMUPDATE statement on table '%s' expected to update %d row(s); %d were matched.MDialect %s does not support updated rowcount - versioning cannot be verified.)N)!r7rj_compiled_cache_memorr: _annotaterTr%r+rimplicit_returningrhZupdate_returningZ_server_onupdate_default_cols!_version_id_has_server_side_valuesupports_sane_rowcountsupports_sane_multi_rowcountexecuter _postfetchcontextcompiled_parametersreturned_defaultsrowcountlen executemanyrStaleDataError descriptionrrOdialect_description)r+r-rr3r5rrtrrr cached_stmtrZ paramkeyshasvaluersrrrecordsrows statementrTassert_singlerowassert_multirowallow_executemanyrrrlrmrZcheck_rowcount multiparamsrrrr(sB                 r()ruse_orm_insert_stmtrc% Cs|dk r,|}tj} t|j} | r&|jp(|} n$d} |} |d|f|j}d|ji} |rdtj| |}n| }d} t |ddD]\\} }}}}}|}|dk r| ||d}|r|s| | j |r|j r| j jsx| sx|rx|sxt|}dd |D}| j|||d }|rt||jjD]X\\}}}}}}}}}|rft||||||||d|jjs^|jnd n t|||qqzt|}| s|j r|st|d kr| r| j js| s| j jrd }n8| r| rd }nd}td| j jd|dnd}nd}|dkr<|s<| | j |r<|j|j|d|i}|j dk rZ|j|j |d}n|rr|j|j!d|i}|rdd |D}| j|||d }|dk r| dkr|} n | "|} |rt#||jj|j$|j%pdD]\\}}}}}}}}}}} |dkrt&'d|t||j(|D]0\}!}"|j)|"}#|*|#j+dkr |!||#j+<q |rtt||||||||d| n t|||qqz| rt,|D]\}}}}} }}}|r| j|-|||d }n| j|||d }|j.}$|$dkrt&'d|ft|$|j(|D]:\}!}"|j)|"}#|"|ks4|*|#j+dkr|!||#j+<q|r|rt|||||||jjd|d|jjsz|jnd n t|||qqz|dk r| dkrt/0S| SdS)z`Emit INSERT statements corresponding to value lists collected by _collect_insert_commands().NFr4rcSs*|dt|dt|d|d|dfSrrrrrrrs   z)_emit_insert_statements..)Z_emit_insert_tableZ_emit_insert_mappercSsg|] }|dqSrrrrrrr[sz+_emit_insert_statements..rrTz+ with RETURNING and sort by parameter orderz with RETURNINGz.update_stmtr?cSs|dt|dfS)Nr)r`rrrrr3z._emit_post_update_statements..rrcSsg|]\}}}}}|qSrr)rrrrrrlrrrr[Ss z0_emit_post_update_statements..rrr)rr7rjrrrTrr:rhrrr_postfetch_post_updaterrrrrrrrrOr)r+r-rr3r5rrrrKrrrrrrrrrrrlrZrrrrr=s         r=cs`jdk ojjkfdd}|df|}t|ddD]\}}dd|D} d |ji} t| } d } d } r|jjs|jjrd } | D]}|j ||| d }| |j 7} qqt d|jj |j || | d n|j || | d }sd} |j } |jrH| d krH| | krH|jjs&t| dkrH| rDt dj| | fqHtdj| | fqHdS)z`Emit DELETE statements corresponding to value lists collected by _collect_delete_commands().Ncslttj}jD] }||tj|j|j dkqr^|j tjj jj j dk |Sr) rrrrr%rrrrKrr7rGr)rrqrZneed_version_idr3rr delete_stmts z,_emit_delete_statements..delete_stmtrGcSs|dS)Nrrrrrrrrz)_emit_delete_statements..cSsg|] \}}|qSrr)rrlrrrrr[sz+_emit_delete_statements..rFrrzMDialect %s does not support deleted rowcount - versioning cannot be verified.TrzDELETE statement on table '%s' expected to delete %d row(s); %d were matched. Please set confirm_deleted_rows=False within the mapper configuration to prevent this warning.)r7rjrrrrrhrrrrrrOrZconfirm_deleted_rowsrrr)r+r-rr3rGrrrZrecsZ del_objectsrexpectedZ rows_matchedZ only_warnrlrZrrrrD|s~      rDc s,|D] \}}}}|jrFfdd|jD}|rFj|g}|jdkrh|j|j|j dk r|j dkr|j j j kr||j j g|r|_ t|t} tj|j| j |d|s|j||n|j|||j dkr|j dk r||j j dkrtdqdS)zzfinalize state on states that have been inserted or updated, including calling after_insert/after_update events. csBg|]:}|jr |jr8|jjks8|js|js|jjkr|jqSr)Zexpire_on_flushdeferredrKrP)rprrrr[s  z4_finalize_insert_update_commands..TNF)Z refresh_stateZonly_load_propsz2Instance does not contain a non-NULL version value)Z_readonly_propsZunmodified_intersection_expire_attributesrPrextendZ_unloaded_non_objectraZ&_server_default_plus_onupdate_propkeysr7riZ_version_id_proprKZunloadedrLr selectZset_label_stylerr Z load_on_identrMrEZ after_insertZ after_updaterr) r+r-r,rrrr1readonlyZ toload_nowrrrrr*sZ     r*c sjdk ojj|k}||s:|jjj}|jjj} n|rHd}} ndS|r`t|jg}tj j j } | rvg} |D]D} | j |krz| j krz|| j |j | j <| rz| j | j qz| r| rڈj j ||| | r||jfdd| DdS)Nrcs"g|]}|jkrj|jqSrrrKrYrrrr[Ts z*_postfetch_post_update..)r7rjrNrcompiledprefetch postfetchr:rJ class_managerrE refresh_flushrKrr"rrP) rr-r3rr0rrlr prefetch_colspostfetch_colsrload_evt_attrsrZrrrr*s@      rc  s|jjj} |jjj} |jjjjdk rHjj|krHt| jg} tj j j } | r^g} r| }|dk rt |D]B\}}|j r|jjrqxj|}|rx|||j<| rx| |jqx| D]N}|j|kr|jkrj|j}||j||<|j|d| r| |q| r.| r.j j ||| |rR|rR| fdd|D| rt||jfdd| Dj|D]"\}}t||||||jq~dS)zExpire attributes in need of newly persisted database state, after an INSERT or UPDATE statement has proceeded for that state.Ncsg|]}|jr|kr|qSr)r)rrq)returning_colsrrr[sz_postfetch..cs"g|]}|jkrj|jqSrrrYrrrr[s )rrrrZeffective_returningr7rjr:rJrrErrrZisinsertrryrKr"r}rrrrPrr rr)rr-r3rr0rrlrmZisupdaterrrrrrowZ row_valuerqrrZpkeyrrr)rrrr\st            rcCs&|j|D]\}}t|||q dSr6)rr Zbulk_populate_inherit_keys)rr0r3rrrrrrsrccs`|jjr|jj}n|j|}d}t||D].}|rB|||}|jj}||j||fVq,dS)zReturn an iterator of (state, state.dict, mapper, connection). The states are sorted according to _sort_states, then paired with the connection they should be using for the given unit of work transaction. N) rMconnection_callableZ transactionrrobjr~rrP)r+r-r,rrrrrrrrIs   rIc Cs|t|}dd|D}||zt||jd}Wn2tk rd}ztd||W5d}~XYnXt|tdd|S)NcSsh|]}|jdk r|qSr6rX)rsrrr s z_sort_states..rXz^Could not sort objects by primary key; primary key values must be sortable in Python (was: %s)Z insert_order) r`difference_updatesortedZ_persistent_sortkey_fn TypeErrorrroperator attrgetter)rr,pending persistentZpersistent_sortederrrrrrs(  r)F)0__doc__ __future__r itertoolsrrrrrrrr r baser rr rrZenginerrrZ sql.elementsrZsql.selectablerr r?rHr!r;r@r&r'r<rCr(r)r=rDr*rrrrIrrrrr sn                  S-+P  j R6- r FvcK2j