Danversbank Deferred Compensation Plan (Amended and Restated as of January 1, 2005)

EXHIBIT 10.4

DANVERSBANK

DEFERRED COMPENSATION PLAN

Amended and Restated as of January 1, 2005

        1.     Danversbank (the "Bank"), a Massachusetts community bank, a wholly-owned subsidiary of Danvers Bancorp, Inc., a mutual form of organization (the "Company"), previously established a Deferred Compensation Plan (the "Plan") for the benefit of specified officers of the Bank who are participants in the Danvers Savings Bank Phantom Stock Plan (the "Phantom Stock Plan") as of January 1, 2003. The Plan is hereby amended and restated as of January 1, 2005.

        2.     Eligibility in the Plan shall be limited to participants in the Phantom Stock Plan and other individuals designated by the Committee from time to time (the "Participants").

        3.     A Participant who wishes to defer the receipt of the distribution of his vested shares from the Phantom Stock Plan must file a written deferral election with the Bank no later than March 15, 2005 with respect to any phantom shares outstanding under the Phantom Stock Plan on March 15, 2005, and no later than December 31 of the calendar year preceding the date of grant with respect to any phantom shares granted after March 15, 2005. Each Participant who is an employee who wishes to elect to defer the receipt of any performance-based cash bonus must file a written deferral election with the Bank no later than June 30 of the calendar year in which the performance-based cash bonus is earned. Any bonus deferral shall be in increments of 10 percent. Such election shall also indicate the timing and method of distribution from this Plan. The amount deferred by each Participant shall be credited to a separate book account (a "Deferred Compensation Account") in his name as of January 1 of the calendar year in which the shares in the Phantom Stock Plan would have vested or as of the date the cash bonus would otherwise have been paid.

        4.     The amount standing to the credit of a Participant's Deferred Compensation Account shall be deemed invested in a savings account earning interest per annum at the "Subject Rate" which shall be the average yield rate on the Bank's portfolio for the preceding calendar year. At the end of each calendar quarter, the amount standing to the credit of a Participant's Deferred Compensation Account shall be adjusted on an equitable basis for deemed interest at the Subject Rate. The reasonable determination of such adjustment by the Committee shall be conclusive and binding on all Participants and their beneficiaries.

        5.     A Participant may elect to receive payment from his Deferred Compensation Account either at a date certain (for example, January 2010) which is at least five (5) calendar years and no more than ten (10) calendar years after his deferral date or upon his termination of employment from the Bank. If there is no election on file, or if the Participant's termination of employment occurs before the date certain, the distribution will be made within 60 days after the Participant's termination of employment from the Bank. Distribution will be in cash and will be payable either in a lump sum or in annual installments over a period not exceeding ten (10) years, except that if the Participant has elected distribution at a date certain, it will be paid only in a lump sum. If there is no election on file, distribution will be made in a lump sum. During the installment period, the Participant's Deferred Compensation Account shall continue to receive interest credit pursuant to Paragraph 4. A Participant may change the form and timing of the distribution while employed by or performing service for the Bank but such change (a) must be made at least 12 calendar months prior to the original distribution date, (b) may not take effect for at least 12 months, (c) must defer the first payment for a period of not less than five years from the original distribution, and (d) may not result in the acceleration of any distribution. Notwithstanding the foregoing, during 2005, a Participant may elect to receive payment of his Deferred Compensation Account in a lump sum, provided, however, that such payment shall be made in 2005. Notwithstanding the following, if a Participant is a "specified employee" within the meaning of Section 409A of the Internal Revenue Code of 1986, as amended, any distribution to the Participant on account of the Participant's separation from service must not be made before the earlier

of (i) six (6) months and a day after the Participant's separation from service, or (ii) the Participant's death. Any payments that would have been made after the Participant's separation from service but for the operation of the preceding sentence shall be made in a lump sum on the date provided in the preceding sentence.

        6.     During his period of active employment, no Participant shall have any rights to the amounts which he has been awarded hereunder. Participation in the Plan, and any actions taken pursuant to the Plan, shall not create or be deemed to create a trust or fiduciary relationship of any kind between the Bank and the Participant. The Bank may, but shall have no obligation to, establish any separate fund, reserve, or escrow or to provide security with respect to any amounts awarded under the Plan. Any assets of the Bank which are set aside in any separate fund, reserve or escrow shall continue for all purposes to be a part of the general assets of the Bank, with title to the beneficial ownership of any such assets remaining at all times in the Bank. No Participant, his legal representatives, or any of his beneficiaries shall have any right, other than the right of an unsecured general creditor of the Bank, in respect of the Deferred Compensation Account established hereunder, and such persons shall have no property interest whatsoever in any specific assets of the Bank.

        7.     Any distribution of deferred compensation payments will be reduced by the amounts required to be withheld pursuant to any governmental law or regulation with respect to taxes or similar provisions. Deferred compensation shall be subject to employment taxes and the Participant shall make arrangements to provide any required withholding amount to the Bank.

        8.     If a Participant who has deferred compensation under the Plan dies before he has received full payment of the amount credited to his account, such unpaid portion shall be paid to the Participant's beneficiary as designated by the Participant in writing. If no beneficiary has been designated or if a designated beneficiary has predeceased the Participant, such unpaid portion shall be paid to the Participant's surviving spouse, if any; otherwise to the Participant's estate.

        9.     The deferred compensation payable under this Plan shall not be subject to alienation, assignment, garnishment, execution, or levy of any kind, and any attempt to cause any compensation to be so subjected shall not be recognized.

        10.   All expenses incurred, or taxes paid by the Bank, and attributable to a Participant's Deferred Compensation Account shall be borne by the Bank and shall not reduce the amount credited to such Deferred Compensation Account.

        11.   Nothing in this Plan shall be construed as giving any Participant the right to be retained in the employ of the Bank in any capacity. The Bank expressly reserves the right to dismiss any employee, at any time, without liability for the effect which such dismissal may have upon such employee hereunder.

        12.   This Plan may be amended in any way or may be terminated, in whole or in part, at any time, and from time to time, by the Board. The foregoing provisions of this paragraph notwithstanding, no amendment or termination of the Plan shall adversely affect the amounts payable hereunder on account of compensation awarded under the Plan prior to the effective date of such amendment or termination or accelerate the time of any distribution under the Plan except to the extent permitted under Section 409A of the Internal Revenue Code of 1986, as amended.

        13.   The Plan shall be administered by the Committee appointed to administer the Phantom Stock Plan. The Committee shall have full power and authority to interpret, construe, and administer this Plan, and the Committee's interpretations and construction thereof, and actions hereunder, including any determination of any amount credited or charged to the Participant's Deferred Compensation Account or the amount or recipient of any payment to be made therefrom, shall be binding and conclusive on all persons for all purposes.

2

        14.   All notices, elections, or designations by a Participant to the Bank shall be delivered in person or by registered mail, postage prepaid, and noted to be brought to the attention of the Committee.

        15.   The terms of this Plan shall be binding upon and shall inure to the benefit of the Bank and its successors or assigns and each Participant and his beneficiaries, heirs, executors, and administrators.

        16.   Subject to its obligation to pay the amount credited to the Participant's Deferred Compensation Account at the time distribution is called, neither the Bank, any person acting on behalf of the Bank nor the Committee shall be liable for any act performed or the failure to perform any act with respect to the terms of the Plan, except in the event that there has been a judicial determination of willful misconduct on the part of the Bank, such person or the Committee.

        17.   This Plan, and all actions taken hereunder, shall be governed by and construed in accordance with the laws of the Commonwealth of Massachusetts, except as such laws may be superseded by any applicable Federal laws.

        18.   This Plan shall be effective as of January 1, 2003. The amendment and restatement shall be effective as of January 1, 2005.

        19.   (a) If a Participant, Beneficiary or their authorized representative (hereinafter the "Claimant") asserts a right to a benefit under the Plan (other than a disability benefit) which has not been received, the Claimant must file a claim for such benefit with the Committee on forms provided by the Committee. The Committee shall render its decision on the claim within 90 days after its receipt of the claim.

        If special circumstances apply, the 90-day period may be extended by an additional 90 days, provided that written notice of the extension is provided to the Claimant during the initial 90-day period and such notice indicates the special circumstances requiring an extension of time and the date by which the Committee expects to render its decision on the claim.

        (b)   If the Committee wholly or partially denies the claim, the Committee shall provide written notice to the Claimant within the time limitations of the immediately preceding paragraph. Such notice shall set forth:

        (i)    the specific reasons for the denial of the claim;

        (ii)   specific reference to pertinent provisions of the Plan on which the denial is based;

        (iii)    a description of any additional material or information necessary to perfect the claim and an explanation of why such material or information is necessary;

        (iv)  a description of the Plan's claims review procedures, and the time limitations applicable to such procedures; and

        (v)   a statement of the Claimant's right to bring a civil action under Section 502(a) of the Employee Retirement Income Security Act of 1974, as amended ("ERISA") if the claim denial is appealed to the Committee and the Committee fully or partially denies the claim.

        (c)   A Claimant whose application for benefits is denied may request a full and fair review of the decision denying the claim by filing, in accordance with such procedures as the Committee may establish, a written appeal which sets forth the documents, records and other information relating to the claim within 60 days after receipt of the notice of the denial from the Committee. In connection with such appeal and upon request by the Claimant, a Claimant may review (or receive free copies of) all documents, records or other information relevant to the Claimant's claim for benefit, all in accordance with such procedures as the Committee may establish. If a Claimant fails to file an appeal within such 60-day period, he shall have no further right to appeal.

3

        (d)   A decision on the appeal by the Committee shall include a review by the Committee that takes into account all comments, documents, records and other information submitted by the Claimant relating to the claim, without regard to whether such information was submitted or considered in the initial claim determination. The Committee shall render its decision on the appeal not later than 60 days after the receipt by the Committee of the appeal. If special circumstances apply, the 60-day period may be extended by an additional 60 days, provided that written notice of the extension is provided to the Claimant during the initial 60-day period and such notice indicates the special circumstances requiring an extension of time and the date by which the Committee expects to render its decision on the claim on appeal.

        If the Committee wholly or partly denies the claim on appeal, the Committee shall provide written notice to the Claimant within the time limitations of the immediately preceding paragraph. Such notice shall set forth:

        (i)    the specific reasons for the denial of the claim;

        (ii)   specific reference to pertinent provisions of the Plan on which the denial is based;

        (iii)    a statement of the Claimant's right to receive, upon request and free of charge, reasonable access to, and copies of, all documents, records, and other information relevant to the Claimant's claim for benefits; and

        (iv)  a statement of the Claimant's right to bring a civil action under Section 502(a) of ERISA.

        The foregoing claims procedures described in this Paragraph 19 shall be administered in accordance with Section 503 of ERISA and guidance issued thereunder. Any written notice required to be given to the Claimant may, at the option of the Committee and in accordance with guidance issued under Section 503 of ERISA, be provided electronically.

        Executed this            day of                        , 2005 by a duly authorized officer of Danversbank.

4

QuickLinks