CONSULTING AGREEMENT

Exhibit 10.4

CONSULTING AGREEMENT

THIS CONSULTING AGREEMENT (this “Agreement”) is effective as of June 1, 2011, by and between PrivateBancorp, Inc. (the “Company”) and Ralph B. Mandell (“Consultant”).

RECITALS

WHEREAS, Consultant retired as an employee of the Company effective May 31, 2011;

WHEREAS, Consultant has unique insights and perspectives from 46 years in the banking industry, including long service as the chief executive officer and a director of the Company; and

WHEREAS, in order to continue to facilitate the smooth transition of management and business and client relationships, the Company desires to engage Consultant over a transition period to be available to provide services as may be requested from time to time and Consultant is willing to provide such consulting services as reasonably requested in advance upon the terms and conditions of this Agreement.

AGREEMENTS

The Company and Consultant hereby agree as follows:

1. Consulting Services.

(a) The Company retains Consultant to consult with and counsel members of management and other managing directors of the Company or The PrivateBank and Trust Company (“Bank”) from time to time as reasonably requested by the Company or the Bank in connection with client relationships, banking services, investor relations or such other business matters as may be reasonably requested from time to time, including participation in meetings of the “advisory boards” established in select geographic markets to the extent requested by senior management (collectively, “Services”).

(b) Consultant agrees to make himself available during normal business hours as reasonably requested in advance and to devote his reasonable best efforts exclusively to the performance of the Services during the term of this Agreement, it being understood and agreed that Consultant will not provide services to other banking organizations during the term of this Agreement.

2. Term. This Agreement will commence as of the date set forth above and will terminate on the earliest to occur of (a) December 31, 2012, (b) the Company’s receipt of written notice of termination by Consultant, (c) Consultant’s receipt of written notice of termination by the Company following Consultant’s breach of this Agreement or his continuing obligations under his employment agreement with the Company dated December 14, 2007 (as amended), including without limitation restrictive covenant agreements, or following his removal as a director of the Company, or (d) Consultant’s Disability or death. For purposes of this Agreement, “Disability” means Consultant’s inability to carry on his duties hereunder or as a director of the Company and the Bank, after receipt of medical advice, as a result of a physical or mental injury or illness or other such incapacity.

3. Fees; Reimbursement of Expenses. The Company will pay Consultant $15,000 per month for the Services hereunder. Each such payment will be made in arrears on or about the last business day of each month during the term of this Agreement. The Company will also reimburse Consultant for reasonable expenses (including parking) incurred in connection with providing Services hereunder, such reimbursement to be made promptly following the Company’s review of expense reports documented by Consultant in accordance with Company policies and procedures.

4. Office and Administrative Support. For the Company’s convenience and to facilitate effective delivery of Services hereunder, the Company will make available suitable office space for Consultant’s use within its existing facilities and will provide appropriate administrative support relating to the provision of Services hereunder.

5. Independent Contractor. Notwithstanding any provision hereof, Consultant is an independent contractor and is not an employee, agent or partner of the Company. Consultant will be solely responsible for the manner and work hours in which the Services are performed under this Agreement. Consultant will not be eligible to participate in any of the Company’s employee benefit plans, fringe benefit programs, group insurance arrangements or similar employee programs (other than as permitted due to his role as a director or prior service as employee). The Company will not provide workers’ compensation, disability insurance, Social Security or unemployment compensation coverage or any other statutory benefit to Consultant, except to the extent such amount or benefit relates to Consultant’s prior service as an employee of the Company. Consultant will comply at Consultant’s expense with all applicable United States federal, state and local laws and regulations required to be fulfilled by independent contractors. Notwithstanding anything to the contrary set forth herein, as a retired employee of the Company, Consultant will be eligible to participate in the Company’s generally applicable post-employment benefit programs and nothing herein is to be construed as limiting compensation for services as a director.

6. Confidentiality and Non-Use of Proprietary Information.

(a) Consultant recognizes and acknowledges the confidential nature of the Proprietary Information of the Company and the damage that could result if such information is disclosed to any third party. “Proprietary Information” means trade secrets, financial information and other proprietary information concerning the products, processes, services or business of the Company or of any clients or prospective clients, which information (i) has not been made generally available to the public, and is useful or of value to the Company’s current or anticipated business activities or of those of any client of the Company; or (ii) has been identified to Consultant as confidential, either orally or in writing, which information includes, but is not limited to: computer programs; research and other statistical data and analyses; marketing, organizational or other research and development, or business plans; personnel information, including the identity of other employees of the Company, their responsibilities, competence, abilities and compensation; financial, accounting and similar records of the Company and/or any fund or account managed by the Company; current and prospective client lists and information on clients and their employees; client financial statements, investment objectives, the nature of their investment portfolios and contractual agreements with the Company, other personal client information and information concerning planned or pending investment products, acquisitions or divestitures. Proprietary Information shall not include information which: (A) is in or hereafter enters the public domain through no fault of Consultant or (B) is obtained by Consultant from a third party having the legal right to use and disclose the same.

(b) Consultant agrees not to use any Proprietary Information except in connection with his duties as a director of the Company and the Bank and his performance of Services hereunder, nor to disclose any Proprietary Information except to officers or other employees of the Company with a need to know such information and when it is appropriate, in the ordinary course of business, to do so. Upon termination of this Agreement, Consultant shall not, directly or indirectly, disclose, publish, communicate or use on his behalf or another’s behalf, any Proprietary Information.

(c) In the event that Consultant is required (in connection with legal proceedings, by oral motion, interrogatory, request for information or documents, subpoena, civil investigation, demand or similar process) to disclose any Proprietary Information, Consultant will provide the Company with prompt notice of such request so that an appropriate protective order can be sought, if deemed necessary.

-2-

(d) The provisions of this Section 6 will survive the termination of this Agreement for any reason.

7. Notices. Any and all notices or other communications required or permitted to be delivered hereunder will be deemed properly delivered if (a) delivered personally, (b) mailed by first class, registered or certified mail, return receipt requested, postage prepaid or (c) sent by next-day or overnight mail or delivery, to the parties as set forth below:

Either party may change the name and address of the designee to whom notice will be sent by giving written notice of such change to the other party.

8. Binding Effect. This Agreement will be binding upon and will inure to the benefit of the undersigned parties, the successors and assigns of the Company (including, but not limited to, any company which may acquire the stock of the Company, all or substantially all of the assets or business of the Company or with or into which the Company may be consolidated or merged) and the heirs, executors, administrators, legal representatives and assigns of Consultant; provided, that the rights and obligations of Consultant hereunder may not be delegated or assigned.

9. Entire Agreement. This Agreement supersedes any former oral agreement and any former written agreement heretofore executed relating generally to the consulting relationship of Consultant with the Company, and this Agreement can only be amended or altered and its provisions can only be waived by an agreement in writing signed by Consultant and the Company. Other than as stated in the preceding sentence, this Agreement does not serve to alter or amend any other agreement between Consultant and the Company or the Bank.

10. Enforceability. This Agreement will be construed and enforced under the laws of the State of Illinois without giving effect to the principles of conflicts of laws thereof. If any provision of this Agreement is held invalid or unenforceable by operation of law or otherwise, such circumstances will not have the effect of rendering any of the other provisions of this Agreement invalid or unenforceable.

11. Counterparts. This Agreement may be executed in counterparts, each of which will be deemed an original but all of which will constitute one and the same instrument. One or more counterparts of this Agreement may be delivered by facsimile, with the intention that delivery by such means will have the same effect as delivery of an original counterpart thereof.

-3-

The parties have executive this Agreement as of the date first written above.

-4-