EMPLOYMENT PROTECTION AGREEMENT

EXHIBIT 10.4

EMPLOYMENT PROTECTION AGREEMENT

This EMPLOYMENT PROTECTION AGREEMENT (the “Agreement”), made as of this 26th day of August, 2003, by and among SBU Bank, a Federally chartered savings bank having its principal place of business in Utica, New York (the “Bank”), Partners Trust Financial Group, Inc., a Federal corporation owning all of the issued and outstanding shares of capital stock of the Bank (“PRTR”) (the Bank and PRTR, collectively, the “Employers”), and Daniel J. O’Toole, an individual residing in Whitesboro, New York (the “Employee”).

WHEREAS, the Employee is currently serving as the Senior Vice President and Chief Credit Officer of the Bank; and

WHEREAS, the Boards of Directors of the Employers (the “Boards”) believe that it is in the best interests of the Employers to encourage the Employee’s continued employment with and dedication to the Bank in the face of potentially distracting circumstances arising from the possibility of a change of control of the Employers, although no such change is now contemplated; and

WHEREAS, the Boards have approved and authorized the entry into this Agreement with the Employee; and

WHEREAS, the parties desire to enter into this Agreement setting forth the terms and conditions for the payment of special compensation to the Employee in the event of a termination of the Employee’s employment in connection with or as the result of a change of control of the Employers;

NOW, THEREFORE, it is AGREED as follows:

1. Term. The initial term of this Agreement shall be for a three-year period commencing on the date of this Agreement. Subject to annual review and approval by the Boards, this Agreement may be renewed by written notice to the Employee for one additional year on the first and each subsequent anniversary of the date of this Agreement. References herein to the term of this Agreement shall include the initial term and any additional years for which this Agreement is renewed.

2. Termination of Employment in Connection with a Change of Control.

(a) If during the term of this Agreement there is a “Change of Control” (as defined below) and the Employee’s employment is terminated, voluntarily by the Employee with “Good Reason” (as defined below) or involuntarily without “Cause” (as defined below), in either case (i) within one year after such

Change of Control, (ii) concurrently therewith or (iii) before and in connection with such Change of Control (including, without limitation, after PRTR (as referred to for purposes of paragraph 2(d) below) has entered into an agreement or engaged in substantive negotiations with respect to such Change of Control) then, subject to Sections 2(e) and (f) below, the following shall apply:

(i) Concurrently with such termination of employment, the Employers shall pay to the Employee a lump sum cash payment equal to (A) the Employee’s base cash compensation from the Bank in effect immediately before the Change of Control or (B) the annual base cash compensation paid to the Employee by the Bank during the calendar year preceding the year in which the Change of Control occurs; and

(ii) For one year after such termination, the Employers shall continue in effect all medical, prescription, dental and life insurance plans for the benefit of the Employee and, if applicable, the Employee’s family, which would have been provided to them if the Employee’s employment had not been terminated; provided, however, that if the Employee becomes reemployed with another employer and is eligible to receive medical, prescription or dental benefits under another employer provided plan, the medical, prescription and dental benefits described herein shall be secondary to those provided under such other plan during such applicable period of eligibility.

Payments under this Section 2(a) shall be in lieu of any amount that may be otherwise owed to the Employee as damages for the loss of employment. Payments under this Section 2(a) shall not be reduced by any compensation which the Employee may receive from other employment with another employer after termination of the Employee’s employment with the Bank. No payment hereunder shall affect the Employee’s entitlement to any vested retirement benefits or other compensation payments.

(b) For purposes of this Agreement, “Good Reason” shall mean (i) a material reduction in the Employee’s position, authority, duties, responsibilities; (ii) a reduction in the Employee’s compensation or perquisites; or (iii) any requirement by the Bank without the written consent of the Employee that the Employee relocate to a place more than 60 miles from Utica, New York to perform his or her duties. To establish that a voluntary termination was with Good Reason, the Employee shall provide the Employers with a written notice of resignation setting out the reasons why the Employee believes that Good Reason exists. Unless the Bank, within 30 days of the date of such notice of resignation, shall reject the Employee’s statement that Good Reason exists, the Employee shall be conclusively deemed to have voluntarily resigned with Good Reason. If the Bank rejects the Employee’s statement that Good Reason exists, the Bank shall have the burden of proving that such rejection of the Employee’s statement was proper.

(c) For purposes of this Agreement, “Cause” shall mean the Employee’s (1) personal dishonesty, incompetence, willful misconduct; (2) breach of fiduciary duty involving personal profit, intentional failure to perform material stated duties; (3) willful violation of any law, rule, or regulation (other than traffic violations or similar offenses); (4) being a specific subject of a final cease and desist order from, written agreement with, or other order or supervisory direction from, any federal or state regulatory authority; or (5) conduct tending to bring the Bank or any of its subsidiaries into substantial public disgrace or disrepute. In determining incompetence, the acts or omissions shall be measured against standards generally prevailing in the financial institutions industry; provided, it shall be the burden of the Employers to prove the alleged acts and omissions and the prevailing nature of the standards the Employers shall have alleged are violated by such acts and/or omissions.

(d) For purposes of this Agreement, a “Change of Control” shall mean:

(1) The acquisition by any individual, entity or group (within the meaning of Section 13(d) (3) or 14(d) (2) of the Securities Exchange Act of 1934, as amended (the “Exchange Act”)) (a “Person”) of beneficial ownership (within the meaning of Rule 13d-3 promulgated under the Exchange Act) of 20% or more of either (i) the then outstanding shares of common stock of PRTR (including, for purposes of this paragraph 2(d), any company into which PRTR may merge as part of the reorganization of Partners Trust, MHC and its affiliates to a converted stock entity) (the “Outstanding Company Common Stock”), or (ii) the combined voting power of the then outstanding voting securities of PRTR entitled to vote generally in the election of directors (the “Outstanding Company Voting Securities”); provided, however, that for purposes of this subsection (1), the following acquisitions shall not constitute a Change of Control: (i) any acquisition directly from PRTR or Partners Trust, MHC, (ii) any acquisition by PRTR or Partners Trust, MHC, (iii) any acquisition by any employee benefit plan (or related trust) sponsored or maintained by PRTR or Partners Trust, MHC, the Bank or any other corporation controlled by PRTR or Partners Trust, MHC, or (iv) any acquisition by any corporation pursuant to a transaction that complies with clauses (i), (ii) and (iii) of subsection (3) of this Section 2(d); or

(2) Individuals who, as of the date of hereof, constitute the Board of Directors of PRTR (the “Incumbent Board”) cease for any reason to constitute at least a majority of such Board of Directors (the “PRTR Board”); provided, however, that any individual becoming a director subsequent to the date of hereof whose election, or nomination for election by PRTR’s shareholders, was approved by a vote of at least a majority of the directors then comprising the Incumbent Board shall be considered as though such individual were a member of the Incumbent Board, but excluding, for this purpose, any such individual whose

initial assumption of office occurs as a result of an actual or threatened election contest with respect to the election or removal of directors or other actual or threatened solicitation of proxies or consents by or on behalf of a Person other than the PRTR Board; or

(3) Consummation of a reorganization, merger or consolidation or sale or other disposition of all or substantially all of the assets of PRTR (a “Business Combination”), in each case, unless, following such Business Combination, (i) all or substantially all of the individuals and entities who were the beneficial owners, respectively, of the Outstanding Company Common Stock and Outstanding Company Voting Securities immediately prior to such Business Combination beneficially own, directly or indirectly, more than 50% of, respectively, the then outstanding shares of common stock and the combined voting power of the then outstanding voting securities entitled to vote generally in the election of directors, as the case may be, of the corporation resulting from such Business Combination (including, without limitation, a corporation which as a result of such transaction owns PRTR or all or substantially all of PRTR’s assets either directly or through one or more subsidiaries) in substantially the same proportions as their ownership, immediately prior to such Business Combination of the Outstanding Company Common Stock and Outstanding Company Voting Securities, as the case may be, (ii) no Person (excluding any corporation resulting from such Business Combination or any employee benefit plan (or related trust) of PRTR, the Bank, such corporation resulting from such Business Combination or a corporation controlled by any of them) beneficially owns, directly or indirectly, 20% or more of, respectively, the then outstanding shares of common stock of the corporation resulting from such Business Combination or the combined voting power of the then outstanding voting securities of such corporation except to the extent that such ownership existed prior to the Business Combination and (iii) at least a majority of the members of the board of directors of the corporation resulting from such Business Combination were members of the Incumbent Board at the time of the execution of the initial agreement, or of the action of the Board, providing for such Business Combination; or

(4) Approval by the shareholders of PRTR of a complete liquidation or dissolution of PRTR without the establishment of a successor corporation.

(e) (1) Notwithstanding any other provision of this Agreement or of any other agreement, contract, or understanding heretofore or hereafter entered into by the Employee and the Employers, except an agreement, contract, or understanding hereafter entered into that expressly modifies or excludes application of this Section 2(e) (the “Other Agreements”), and notwithstanding any formal or informal plan or other arrangement for the direct or indirect provision of compensation to the Employee (including groups or classes of participants or

beneficiaries of which the Employee is a member), whether or not such compensation is deferred, is in cash, or is in the form of a benefit to or for the Employee (a “Benefit Plan”), the Employee shall not have any right to receive any payment or other benefit under this Agreement, any Other Agreement, or any Benefit Plan if such payment or benefit, taking into account all other payments or benefits to or for the Employee under this Agreement, all Other Agreements, and all Benefit Plans, would cause any payment to the Employee under this Agreement to be considered a “parachute payment” within the meaning of Section 280G(b)(2) of the Internal Revenue Code of 1986, as amended (a “Parachute Payment”). In the event that the receipt of any such payment or benefit under this Agreement, any Other Agreement, or any Benefit Plan would cause the Employee to be considered to have received a Parachute Payment under this Agreement, then the Employee shall have the right, in the Employee’s sole discretion, to designate those payments or benefits under this Agreement, any Other Agreements, and/or any Benefit Plans, which should be reduced or eliminated so as to avoid having the payment to the Employee under this Agreement be deemed to be a Parachute Payment.

(2) All determinations required to be made under this Section 2(e), including whether and when the payments and benefits referred to in Section 2(e)(1) shall be reduced, shall be made by KPMG LLP or such other certified public accounting firm as may be designated by the Employee and shall be reasonably acceptable to the Employers (the “Accounting Firm”) which shall provide detailed supporting calculations both to the Employers and the Employee within 15 business days of the receipt of notice from the Employee that such a determination is required, or such earlier time as is requested by the Employers. In the event that the Accounting Firm is serving as accountant or auditor for the individual, entity or group effecting a change in the ownership or effective control (as defined for purposes of Section 280G of the Code) of the Employers, the Employee shall appoint another nationally recognized accounting firm which is reasonably acceptable to the Employers to make the determinations required hereunder (which accounting firm shall then be referred to as the Accounting Firm hereunder). All fees and expenses of the Accounting Firm shall be borne solely by the Employers. Any determination by the Accounting Firm shall be binding upon the Employers and the Employee. As a result of the uncertainty in the application of Section 280G of the Code at the time of the initial determination by the Accounting Firm hereunder, it is possible that additional adjustments may be required. The Employee shall promptly repay to the Employers any amount that is later determined to constitute a Parachute Payment, with interest at 120% of the applicable federal rate, as defined for purposes of Section 280G of the Code and all such amounts shall be treated as loans from the Employers to the Employee.

(f) Notwithstanding any other provision in this Agreement, (i) the Employers may terminate or suspend this Agreement and the employment of the Employee, as if such termination were for Cause, to the extent required by the

applicable laws of the State of New York related to banking, by applicable federal law relating to deposit insurance or bank holding companies or by regulations or orders issued by the New York State Banking Department, the Board of Governors of the Federal Reserve System, the Federal Deposit Insurance Corporation or other state or federal banking regulatory agency having jurisdiction over PRTR or the Bank and (ii) no payment shall be required to be made to or for the benefit of the Employee under this Agreement to the extent such payment is prohibited by applicable law, regulation or order issued by a banking agency or a court of competent jurisdiction; provided, that it shall be the Employers’ burden to prove that any such action was so required.

3. No Assignments. This Agreement is personal to each of the parties hereto. No party may assign or delegate any rights or obligations hereunder without first obtaining the written consent of the other party hereto. However, in the event of the death of the Employee, all rights to receive payments hereunder shall become rights of the Employee’s estate. Subject to the provisions hereof restricting assignment, this Agreement shall be binding upon the parties hereto and shall inure to the benefit of the parties and their respective heirs, devisees, executors, administrators, legal representatives, successors and assigns.

4. Amendments or Additions; Action by Board of Directors. No amendments or additions to this Agreement shall be binding unless in writing and signed by both parties hereto. The prior approval by a majority affirmative vote of the full Boards shall be required in order for the Employers to authorize any amendments or additions to this Agreement.

5. Section Headings. The section headings used in this Agreement are included solely for convenience and shall not affect, or be used in connection with, the interpretation of this Agreement.

6. Governing Law. This Agreement shall be governed by the laws of the United States to the extent applicable, and otherwise by the laws of the State of New York applicable to contracts entered into and performed wholly within its borders.

7. Entire Agreement. This Agreement constitutes the entire agreement between the parties respecting the employment of the Employee, there being no representations, warranties or commitments except as set forth herein.

8. Reserved.

9. Notices. All notices, demands, requests, or other communications which may be or are required to be given, served, or sent by any party to any other party pursuant to this Agreement shall be in writing and shall be hand delivered, sent by overnight courier or mailed by first-class, registered or certified mail, return

receipt requested, postage prepaid, or transmitted by telegram, telecopy or telex, addressed as follows:

Partners Trust Financial Group, Inc.

233 Genesee Street

Utica, NY 13501

Attn: Chief Executive Officer

with a copy (which shall not constitute notice) to:

Stuart G. Stein

Hogan & Hartson, L.L.P.

555 13th Street, N.W.

Washington, DC 20004-1109

Fax: 202 ###-###-####

Daniel J. O’Toole

109 Bedford Drive

Whitesboro, NY 13492

Each party may designate by notice in writing a new address to which any notice, demand, request or communication may thereafter be so given, served or sent. Each notice, demand, request, or communication which shall be hand delivered, sent, mailed, telecopied or telexed in the manner described above, or which shall be delivered to a telegraph company, shall be deemed sufficiently given, served, sent, received or delivered for all purposes at such time as it is delivered to the addressee (with the return receipt, the delivery receipt, or (with respect to a telecopy or telex) the answerback being deemed conclusive, but not exclusive, evidence of such delivery) or at such time as delivery is refused by the addressee upon presentation.

10. Severability. The invalidity or unenforceability of any one or more provisions of this Agreement shall not affect the validity or enforceability of the other provisions of this Agreement, which shall remain in full force and effect.

11. Counterparts. This Agreement may be executed in two or more counterparts, each of which shall be an original and all of which shall be deemed to constitute one and the same instrument.

IN WITNESS WHEREOF, the undersigned have duly executed and delivered this Agreement, or have caused this Agreement to be duly executed and delivered on their behalf, as of the date written above.