Capital Securities Guarantee Agreement First Midwest Bancorp, Inc.

Exhibit 4.3

Capital Securities Guarantee Agreement

First Midwest Bancorp, Inc.

Dated as of August 21, 2009

Table of Contents

-i-

-ii-

Capital Securities Guarantee Agreement

This Guarantee Agreement (the “Capital Securities Guarantee”), dated as of August 21, 2009, is executed and delivered by First Midwest Bancorp, Inc., a Delaware corporation (the “Guarantor”), and Wilmington Trust Company, a Delaware banking corporation, as trustee (the “Capital Securities Guarantee Trustee”), for the benefit of the Holders (as defined herein) from time to time of the Capital Securities (as defined herein) of First Midwest Capital Trust I, a Delaware statutory trust, formerly known as New First Midwest Capital Trust I (the “Issuer”).

Witnesseth:

Whereas, pursuant to an Amended and Restated Declaration of Trust (the “Declaration”), dated as of August 21, 2009, among the trustees of the Issuer, the Guarantor, as sponsor, and the holders from time to time of undivided beneficial interests in the assets of the Issuer and an Agreement of Merger, dated as of August 21, 2009 (the “Agreement of Merger”), between the Issuer and the Predecessor Issuer (as defined herein) the Issuer has outstanding on the date hereof 125,000 capital securities, having an aggregate liquidation amount of $125,000,000, such capital securities being designated the 6.95% Capital Securities (collectively the “Capital Securities”); and

Whereas, the Guarantor desires irrevocably and unconditionally to agree, to the extent set forth in this Capital Securities Guarantee, to pay to the Holders the Guarantee Payments (as defined below) and to make certain other payments on the terms and conditions set forth herein;

Now, Therefore, in consideration of the premises, the Guarantor executes and delivers this Capital Securities Guarantee for the benefit of the Holders.

Article I

Definitions and Interpretation

Section 1.1. Definitions and Interpretation. In this Capital Securities Guarantee, unless the context otherwise requires:

(a) capitalized terms used in this Capital Securities Guarantee but not defined in the preamble above have the respective meanings assigned to them in this Section 1.1;

(b) terms defined in the Declaration as of the date of execution of this Capital Securities Guarantee have the same meaning when used in this Capital Securities Guarantee unless otherwise defined in this Capital Securities Guarantee;

(c) a term defined anywhere in this Capital Securities Guarantee has the same meaning throughout;

(d) all references to “the Capital Securities Guarantee” or “this Capital Securities Guarantee” are to this Capital Securities Guarantee as modified, supplemented or amended from time to time;

(e) all references in this Capital Securities Guarantee to Articles and Sections are to Articles and Sections of this Capital Securities Guarantee, unless otherwise specified;

(f) a term defined in the Trust Indenture Act has the same meaning when used in this Capital Securities Guarantee, unless otherwise defined in this Capital Securities Guarantee or unless the context otherwise requires; and

(g) a reference to the singular includes the plural and vice versa.

“Affiliate” has the same meaning as given to that term in Rule 405 under the Securities Act of 1933, as amended, or any successor rule thereunder.

“Business Day” means any day other than a Saturday or a Sunday, or a day on which banking institutions in The City of New York are authorized or required by law, executive order or regulation to close or a day on which the Corporate Trust Office of the Capital Securities Guarantee Trustee is closed for business.

“Capital Securities Guarantee Trustee” means Wilmington Trust Company, a Delaware banking corporation, until a Successor Capital Securities Guarantee Trustee has been appointed and has accepted such appointment pursuant to the terms of this Capital Securities Guarantee and thereafter means each such Successor Capital Securities Guarantee Trustee.

“Common Securities” means the securities representing common undivided beneficial interests in the assets of the Issuer.

“Corporate Trust Office” means the office of the Capital Securities Guarantee Trustee at which the corporate trust business of the Capital Securities Guarantee Trustee shall, at any particular time, be principally administered, which office at the date of execution of this Agreement is located at Rodney Square North, 1100 North Market Street, Wilmington, Delaware ###-###-####, Attention: Corporate Trust Administration.

“Covered Person” means any Holder or beneficial owner of Capital Securities.

“Debentures” means the series of subordinated debt securities of the Guarantor designated the 6.95% Series B Junior Subordinated Deferrable Interest Debentures due December 1, 2033 held by the Property Trustee (as defined in the Declaration) of the Issuer.

“Event of Default” means a default by the Guarantor on any of its payment or other obligations under this Capital Securities Guarantee.

“Guarantee Payments” means the following payments or distributions, without duplication, with respect to the Capital Securities, to the extent not paid by or on behalf of the Issuer: (i) any accumulated and unpaid Distributions (as defined in the Declaration) that are required to be paid on such Capital Securities, to the extent the Issuer has funds on hand legally

-2-

available therefor at such time, (ii) the redemption price, including all accumulated and unpaid Distributions to the date of redemption (the “Redemption Price”), to the extent the Issuer has funds on hand legally available therefor at such time, with respect to any Capital Securities called for redemption by the Issuer, and (iii) upon a voluntary or involuntary dissolution, winding up or liquidation of the Issuer (other than in connection with the distribution of Debentures to the Holders in exchange for Capital Securities as provided in the Declaration), the lesser of (a) the aggregate of the liquidation amount and all accumulated and unpaid Distributions on the Capital Securities to the date of payment, to the extent the Issuer has funds on hand legally available therefor, and (b) the amount of assets of the Issuer remaining available for distribution to Holders upon liquidation of the Issuer after satisfaction of liabilities to creditors of the Issuer as required by applicable law.

“Holder” shall mean any holder, as registered on the books and records of the Issuer, of any Capital Securities; provided, however, that, in determining whether the holders of the requisite percentage of Capital Securities have given any request, notice, consent or waiver hereunder, “Holder” shall not include the Guarantor or any Affiliate of the Guarantor.

“Indemnified Person” means the Capital Securities Guarantee Trustee, any Affiliate of the Capital Securities Guarantee Trustee, or any officers, directors, shareholders, members, partners, employees, representatives, nominees, custodians or agents of the Capital Securities Guarantee Trustee.

“Indenture” means the Indenture dated as of November 18, 2003, as amended and supplemented, among the Guarantor (the “Debenture Issuer”) and Wilmington Trust Company, as trustee, pursuant to which the Debentures are issued and outstanding.

“Majority in liquidation amount of the Capital Securities” means, except as provided by the Trust Indenture Act, a vote by Holder(s) of more than 50% of the aggregate liquidation amount (including the stated amount that would be paid on redemption, liquidation or otherwise, plus accumulated and unpaid Distributions to the date upon which the voting percentages are determined) of all outstanding Capital Securities.

“Officers’ Certificate” means, with respect to the Guarantor, a certificate signed by the Chairman, the Chief Executive Officer, the President, a Vice President, the Chief Financial Officer, the Secretary or an Assistant Secretary, of the Guarantor. Any Officers’ Certificate delivered with respect to compliance with a condition or covenant provided for in this Capital Securities Guarantee (other than pursuant to Section 314(a)(4) of the Trust Indenture Act) shall include:

(a) a statement that each officer signing the Officers’ Certificate has read the covenant or condition and the definitions relating thereto;

(b) a brief statement of the nature and scope of the examination or investigation undertaken by the officer rendering the Officers’ Certificate;

(c) a statement that each such officer has made such examination or investigation as, in such officer’s opinion, is necessary to enable such officer to express an informed opinion as to whether or not such covenant or condition has been complied with; and

-3-

(d) a statement as to whether, in the opinion of each such officer, such condition or covenant has been complied with.

“Other Debentures” means all junior subordinated debentures issued by the Guarantor from time to time and sold to trusts, in each case similar to the Issuer, to be established by the Guarantor (if any).

“Other Guarantees” means all guarantees to be issued by the Guarantor with respect to capital securities (if any) similar to the Capital Securities issued by other trusts, in each case similar to the Issuer established by the Guarantor (if any).

“Person” means a legal person, including any individual, corporation, estate, partnership, joint venture, association, joint stock company, limited liability company, trust, unincorporated association, or government or any agency or political subdivision thereof, or any other entity of whatever nature.

“Predecessor Issuer” means First Midwest Capital Trust I, a Delaware statutory trust created pursuant to the Delaware Statutory Trust Act by the entering into of that certain Declaration of Trust, dated as of November 6, 2003, as amended and restated in its entirety by the Amended and Restated Declaration of Trust, dated November 18, 2003, and by the execution and filing by Wilmington Trust Company , as Trustee, with the Secretary of State of the State of Delaware of the Certificate of Trust, filed on November 6, 2003.

“Responsible Officer” means, when used with respect to the Capital Securities Guarantee Trustee, any officer of the Capital Securities Guarantee Trustee with direct responsibility for the administration of this Capital Securities Guarantee and also means, with respect to a particular corporate trust matter, any other officer to whom such matter is referred because of that Officer’s knowledge of and familiarity with the particular subject.

“Successor Capital Securities Guarantee Trustee” means a successor Capital Securities Guarantee Trustee possessing the qualifications to act as Capital Securities Guarantee Trustee under Section 4.1.

“Trust Indenture Act” means the Trust Indenture Act of 1939, as amended.

“Trust Securities” means the Common Securities and the Capital Securities, collectively.

Article II

Trust Indenture Act

Section 2.1. Trust Indenture Act; Application. (a) This Capital Securities Guarantee is subject to the provisions of the Trust Indenture Act that are required to be part of this Capital Securities Guarantee and shall, to the extent applicable, be governed by such provisions;

-4-

(b) the Capital Securities Guarantee Trustee shall be the only trustee for purposes of the Trust Indenture Act; and

(c) if and to the extent that any provision of this Capital Securities Guarantee limits, qualifies or conflicts with the duties imposed by Section 310 to 317, inclusive, of the Trust Indenture Act, such imposed duties shall control.

Section 2.2. Lists of Holders of Securities. (a) The Guarantor shall provide the Capital Securities Guarantee Trustee (unless the Capital Securities Guarantee Trustee is otherwise the registrar of the Capital Securities) with a list, in such form as the Capital Securities Guarantee Trustee may reasonably require, of the names and addresses of the Holders (“List of Holders”) as of such date, (i) within one Business Day after May 1 and November 1 of each year, and (ii) at any other time within 30 days of receipt by the Guarantor of a written request for a List of Holders as of a date no more than 14 days before such List of Holders is given to the Capital Securities Guarantee Trustee; provided, that the Guarantor shall not be obligated to provide such List of Holders at any time the List of Holders does not differ from the most recent List of Holders given to the Capital Securities Guarantee Trustee by the Guarantor. The Capital Securities Guarantee Trustee may destroy any List of Holders previously given to it on receipt of a new List of Holders.

(b) The Capital Securities Guarantee Trustee shall comply with its obligations under Sections 311(a), 311(b) and Section 312(b) of the Trust Indenture Act.

Section 2.3. Reports by the Capital Securities Guarantee Trustee. Within 60 days after May 15 of each year, commencing May 15, 2010, the Capital Securities Guarantee Trustee shall provide to the Holders such reports as are required by Section 313(a) of the Trust Indenture Act, if any, in the form and in the manner provided by Section 313 of the Trust Indenture Act. The Capital Securities Guarantee Trustee shall also comply with the other requirements of Section 313 of the Trust Indenture Act.

Section 2.4. Periodic Reports to Capital Securities Guarantee Trustee. The Guarantor shall provide to the Capital Securities Guarantee Trustee such documents, reports and information as required by Section 314 of the Trust Indenture Act, if any, and the compliance certificate required by Section 314 of the Trust Indenture Act in the form, in the manner and at the times required by Section 314(a)(4) of the Trust Indenture Act; provided that such compliance certificate shall be delivered on or before 120 days after the end of each fiscal year of the Guarantor. Delivery of such documents, reports and information to the Capital Securities Guarantee Trustee is for informational purposes only and the Capital Securities Guarantee Trustee’s receipt of such shall not constitute constructive notice of any information contained therein or determinable from information contained therein, including the Guarantor’s compliance with any of its covenants hereunder (as to which the Capital Securities Guarantee Trustee is entitled to rely exclusively on Officers’ Certificates).

Section 2.5. Evidence of Compliance with Conditions Precedent. The Guarantor shall provide to the Capital Securities Guarantee Trustee such evidence of compliance with any conditions precedent, if any, provided for in this Capital Securities Guarantee that relate to any of the matters set forth in Section 314(c) of the Trust Indenture Act. Any certificate or opinion required to be given by an officer pursuant to Section 314(c)(1) may be given in the form of an Officers’ Certificate.

-5-

Section 2.6. Events of Default; Waiver. The Holders of a Majority in liquidation amount of Capital Securities may, by vote, on behalf of all the Holders, waive any past Event of Default and its consequences. Upon such waiver, any such Event of Default shall cease to exist, and any Event of Default arising therefrom shall be deemed to have been cured, for every purpose of this Capital Securities Guarantee, but no such waiver shall extend to any subsequent or other Event of Default or impair any right consequent thereon.

Section 2.7. Event of Default; Notice. (a) The Capital Securities Guarantee Trustee shall, within 60 days after the occurrence of an Event of Default with respect to this Capital Securities Guarantee, transmit by mail, first class postage prepaid, to all Holders, notices of such Event of Default actually known to a Responsible Officer of the Capital Securities Guarantee Trustee, unless such Event of Default has been cured before the giving of such notice; provided, that, except in the case of an Event of Default in the payment of any Guarantee Payment, the Capital Securities Guarantee Trustee shall be protected in withholding such notice if and so long as the board of directors, the executive committee, or a trust committee of directors and/or Responsible Officers of the Capital Securities Guarantee Trustee in good faith determines that the withholding of such notice is in the interests of the Holders of the Capital Securities.

(b) The Capital Securities Guarantee Trustee shall not be deemed to have knowledge of any Event of Default unless the Capital Securities Guarantee Trustee shall have received written notice from the Guarantor or a Holder, or a Responsible Officer of the Capital Securities Guarantee Trustee shall have obtained actual knowledge, of such Event of Default.

Section 2.8. Conflicting Interests. The Declaration and Indenture shall be deemed to be specifically described in this Capital Securities Guarantee for the purposes of clause (i) of the first proviso contained in Section 310(b) of the Trust Indenture Act.

Article III

Powers, Duties and Rights of Capital Securities Guarantee Trustee

Section 3.1. Powers and Duties of the Capital Securities Guarantee Trustee. (a) This Capital Securities Guarantee shall be held by the Capital Securities Guarantee Trustee for the benefit of the Holders, and the Capital Securities Guarantee Trustee shall not transfer this Capital Securities Guarantee to any Person except a Holder exercising his or her rights pursuant to Section 5.4(b) or to a Successor Capital Securities Guarantee Trustee on acceptance by such Successor Capital Securities Guarantee Trustee of its appointment to act as Successor Capital Securities Guarantee Trustee. The right, title and interest of the Capital Securities Guarantee Trustee shall automatically vest in any Successor Capital Securities Guarantee Trustee, and such vesting and succession of title shall be effective whether or not conveyancing documents have been executed and delivered pursuant to the appointment of such Successor Capital Securities Guarantee Trustee.

-6-

(b) If an Event of Default actually known to a Responsible Officer of the Capital Securities Guarantee Trustee has occurred and is continuing, the Capital Securities Guarantee Trustee shall enforce this Capital Securities Guarantee for the benefit of the Holders.

(c) The Capital Securities Guarantee Trustee, before the occurrence of any Event of Default and after the curing of all Events of Default that may have occurred, shall undertake to perform only such duties as are specifically set forth in this Capital Securities Guarantee, and no implied covenants shall be read into this Capital Securities Guarantee against the Capital Securities Guarantee Trustee. In case an Event of Default has occurred (that has not been cured or waived pursuant to Section 2.6) and is actually known to a Responsible Officer of the Capital Securities Guarantee Trustee, the Capital Securities Guarantee Trustee shall exercise such of the rights and powers vested in it by this Capital Securities Guarantee, and use the same degree of care and skill in its exercise thereof, as a prudent Person would exercise or use under the circumstances in the conduct of his or her own affairs.

(d) No provision of this Capital Securities Guarantee shall be construed to relieve the Capital Securities Guarantee Trustee from liability for its own negligent action, its own negligent failure to act, or its own willful misconduct, except that:

(i) prior to the occurrence of any Event of Default and after the curing or waiving of all such Events of Default that may have occurred:

(A) the duties and obligations of the Capital Securities Guarantee Trustee shall be determined solely by the express provisions of this Capital Securities Guarantee, and the Capital Securities Guarantee Trustee shall not be liable except for the performance of such duties and obligations as are specifically set forth in this Capital Securities Guarantee, and no implied covenants or obligations shall be read into this Capital Securities Guarantee against the Capital Securities Guarantee Trustee; and

(B) in the absence of bad faith on the part of the Capital Securities Guarantee Trustee, the Capital Securities Guarantee Trustee may conclusively rely, as to the truth of the statements and the correctness of the opinions expressed therein, upon any certificates or opinions furnished to the Capital Securities Guarantee Trustee and conforming to the requirements of this Capital Securities Guarantee; provided, however, in the case of any such certificates or opinions that by any provision hereof are specifically required to be furnished to the Capital Securities Guarantee Trustee, the Capital Securities Guarantee Trustee shall be under a duty to examine the same to determine whether or not they conform to the requirements of this Capital Securities Guarantee;

(ii) the Capital Securities Guarantee Trustee shall not be liable for any error of judgment made in good faith by a Responsible Officer of the Capital Securities Guarantee Trustee, unless it shall be proved that the Capital Securities Guarantee Trustee was negligent in ascertaining the pertinent facts upon which such judgment was made;

-7-

(iii) the Capital Securities Guarantee Trustee shall not be liable with respect to any action taken or omitted to be taken by it in good faith in accordance with the direction of the Holders of a Majority in liquidation amount of the Capital Securities relating to the time, method and place of conducting any proceeding for any remedy available to the Capital Securities Guarantee Trustee, or exercising any trust or power conferred upon the Capital Securities Guarantee Trustee under this Capital Securities Guarantee; and

(iv) no provision of this Capital Securities Guarantee shall require the Capital Securities Guarantee Trustee to expend or risk its own funds or otherwise incur personal financial liability in the performance of any of its duties or in the exercise of any of its rights or powers, if the Capital Securities Guarantee Trustee shall have reasonable grounds for believing that the repayment of such funds or liability is not reasonably assured to it under the terms of this Capital Securities Guarantee or indemnity, reasonably satisfactory to the Capital Securities Guarantee Trustee, against such risk or liability is not reasonably assured to it.

(e) This Capital Securities Guarantee and all moneys received by the Capital Securities Guarantee Trustee hereunder in respect of the Guarantee Payments will not be subject to any right, charge, security interest, loan or claim of any kind in favor of or for the benefit of the Capital Securities Guarantee Trustee or its agents or creditors.

Section 3.2. Certain Rights of Capital Securities Guarantee Trustee. (a) Subject to the provisions of Section 3.1:

(i) The Capital Securities Guarantee Trustee may conclusively rely, and shall be fully protected in acting or refraining from acting, upon any resolution, certificate, statement, instrument, opinion, report, notice, request, direction, consent, order, bond, debenture, note, other evidence of indebtedness or other paper or document believed by it to be genuine and to have been signed, sent or presented by the proper party or parties.

(ii) Any direction or act of the Guarantor contemplated by this Capital Securities Guarantee shall be sufficiently evidenced by an Officers’ Certificate.

(iii) Whenever, in the administration of this Capital Securities Guarantee, the Capital Securities Guarantee Trustee shall deem it desirable that a matter be proved or established before taking, suffering or omitting any action hereunder, the Capital Securities Guarantee Trustee (unless other evidence is herein specifically prescribed) may, in the absence of bad faith on its part, request and conclusively rely upon an Officers’ Certificate which, upon receipt of such request, shall be promptly delivered by the Guarantor.

(iv) The Capital Securities Guarantee Trustee shall have no duty to see to any recording, filing or registration of any instrument (or any rerecording, refiling or registration thereof).

(v) The Capital Securities Guarantee Trustee may consult with counsel of its selection, and the written advice or opinion of such counsel with respect to legal matters shall be full and complete authorization and protection in respect of any action taken, suffered or omitted by it hereunder in good faith and in accordance with such written advice or opinion. Such counsel may be counsel to the Guarantor or any of its Affiliates and may include any of its employees. The Capital Securities Guarantee Trustee shall have the right at any time to seek instructions concerning the administration of this Capital Securities Guarantee from any court of competent jurisdiction.

-8-

(vi) The Capital Securities Guarantee Trustee shall be under no obligation to exercise any of the rights or powers vested in it by this Capital Securities Guarantee at the request or direction of any Holder, unless such Holder shall have provided to the Capital Securities Guarantee Trustee such security and indemnity, reasonably satisfactory to the Capital Securities Guarantee Trustee, against the costs, expenses (including reasonable attorneys’ fees and expenses and the expenses of the Capital Securities Guarantee Trustee’s agents, nominees or custodians) and liabilities that might be incurred by it in complying with such request or direction, including such reasonable advances as may be requested by the Capital Securities Guarantee Trustee; provided that, nothing contained in this Section 3.2(a)(vi) shall be taken to relieve the Capital Securities Guarantee Trustee, upon the occurrence of an Event of Default which has not been cured or waived, of its obligation to exercise the rights and powers vested in it by this Capital Securities Guarantee and to use the same degree of care as a prudent Person would exercise or use under the circumstances in the conduct of his or her own affairs.

(vii) The Capital Securities Guarantee Trustee shall not be bound to make any investigation into the facts or matters stated in any resolution, certificate, statement, instrument, opinion, report, notice, request, direction, consent, order, bond, debenture, note, other evidence of indebtedness or other paper or document, but the Capital Securities Guarantee Trustee, in its discretion, may make such further inquiry or investigation into such facts or matters as it may see fit.

(viii) The Capital Securities Guarantee Trustee may execute any of the trusts or powers hereunder or perform any duties hereunder either directly or by or through agents, nominees, custodians or attorneys, and the Capital Securities Guarantee Trustee shall not be responsible for any misconduct or negligence on the part of any agent or attorney appointed with due care by it hereunder.

(ix) Any action taken by the Capital Securities Guarantee Trustee or its agents hereunder shall bind the Holders, and the signature of the Capital Securities Guarantee Trustee or its agents alone shall be sufficient and effective to perform any such action. No third party shall be required to inquire as to the authority of the Capital Securities Guarantee Trustee to so act or as to its compliance with any of the terms and provisions of this Capital Securities Guarantee, both of which shall be conclusively evidenced by the Capital Securities Guarantee Trustee’s or its agent’s taking such action.

(x) Whenever in the administration of this Capital Securities Guarantee the Capital Securities Guarantee Trustee shall deem it desirable to receive instructions with respect to enforcing any remedy or right or taking any other action hereunder, the Capital Securities Guarantee Trustee (i) may request instructions from the Holders of a Majority in liquidation amount of the Capital Securities, (ii) may refrain from enforcing such remedy or right or taking such other action until such instructions are received, and (iii) shall be protected in conclusively relying on or acting in accordance with such instructions.

-9-

(xi) The Capital Securities Guarantee Trustee shall not be liable for any action taken, suffered, or omitted to be taken by it in good faith, without negligence, and reasonably believed by it to be authorized or within the discretion or rights or powers conferred upon it by this Capital Securities Guarantee.

(b) No provision of this Capital Securities Guarantee shall be deemed to impose any duty or obligation on the Capital Securities Guarantee Trustee to perform any act or acts or exercise any right, power, duty or obligation conferred or imposed on it in any jurisdiction in which it shall be illegal, or in which the Capital Securities Guarantee Trustee shall be unqualified or incompetent in accordance with applicable law, to perform any such act or acts or to exercise any such right, power, duty or obligation. No permissive power or authority available to the Capital Securities Guarantee Trustee shall be construed to be a duty.

Section 3.3. Not Responsible for Recitals or Issuance of Capital Securities Guarantee. The recitals contained in this Capital Securities Guarantee shall be taken as the statements of the Guarantor, and the Capital Securities Guarantee Trustee does not assume any responsibility for their correctness. The Capital Securities Guarantee Trustee makes no representation as to the validity or sufficiency of this Capital Securities Guarantee.

Article IV

Capital Securities Guarantee Trustee

Section 4.1. Capital Securities Guarantee Trustee; Eligibility. (a) There shall at all times be a Capital Securities Guarantee Trustee which shall:

(i) not be an Affiliate of the Guarantor; and

(ii) be a corporation organized and doing business under the laws of the United States of America or any state or territory thereof or of the District of Columbia, or a corporation or Person permitted to act as a trustee by the Securities and Exchange Commission, authorized under such laws to exercise corporate trust powers, having a combined capital and surplus of at least 50 million U.S. dollars ($50,000,000) and subject to supervision or examination by federal, state, territorial or District of Columbia authority. If such corporation publishes reports of condition at least annually, pursuant to law or to the requirements of the supervising or examining authority referred to above, then, for the purposes of this Section 4.1(a)(ii), the combined capital and surplus of such corporation shall be deemed to be its combined capital and surplus as set forth in its most recent report of condition so published.

(b) If at any time the Capital Securities Guarantee Trustee shall cease to be eligible to so act under Section 4.1(a), the Capital Securities Guarantee Trustee shall immediately resign in the manner and with the effect set forth in Section 4.2(c).

(c) If the Capital Securities Guarantee Trustee has or shall acquire any “conflicting interest” within the meaning of Section 310(b) of the Trust Indenture Act, the Capital Securities Guarantee Trustee and Guarantor shall in all respects comply with the provisions of Section 310(b) of the Trust Indenture Act, subject to the penultimate paragraph thereof.

-10-

Section 4.2. Appointment, Removal and Resignation of Capital Securities Guarantee Trustee. (a) Subject to Section 4.2(b), the Capital Securities Guarantee Trustee may be appointed or removed without cause at any time by the Guarantor except during an Event of Default.

(b) The Capital Securities Guarantee Trustee shall not be removed in accordance with Section 4.2(a) until a Successor Capital Securities Guarantee Trustee has been appointed and has accepted such appointment by written instrument executed by such Successor Capital Securities Guarantee Trustee and delivered to the Guarantor and to the Capital Securities Trustee being removed.

(c) The Capital Securities Guarantee Trustee shall hold office until a Successor Capital Securities Guarantee Trustee shall have been appointed or until its removal or resignation. The Capital Securities Guarantee Trustee may resign from office (without need for prior or subsequent accounting) by an instrument in writing executed by the Capital Securities Guarantee Trustee and delivered to the Guarantor, which resignation shall not take effect until a Successor Capital Securities Guarantee Trustee has been appointed and has accepted such appointment by an instrument in writing executed by such Successor Capital Securities Guarantee Trustee and delivered to the Guarantor and the resigning Capital Securities Guarantee Trustee.

(d) If no Successor Capital Securities Guarantee Trustee shall have been appointed and accepted appointment as provided in this Section 4.2 within 60 days after delivery of an instrument of removal or resignation, the Capital Securities Guarantee Trustee resigning or being removed may petition any court of competent jurisdiction for appointment of a Successor Capital Securities Guarantee Trustee. Such court may thereupon, after prescribing such notice, if any, as it may deem proper, appoint a Successor Capital Securities Guarantee Trustee.

(e) No Capital Securities Guarantee Trustee shall be liable for the acts or omissions to act of any Successor Capital Securities Guarantee Trustee.

(f) Upon termination of this Capital Securities Guarantee or removal or resignation of the Capital Securities Guarantee Trustee pursuant to this Section 4.2, the Guarantor shall pay to the Capital Securities Guarantee Trustee all amounts due to the Capital Securities Guarantee Trustee accrued to the date of such termination, removal or resignation.

Article V

Guarantee

Section 5.1. Guarantee. The Guarantor irrevocably and unconditionally agrees to pay in full to the Holders the Guarantee Payments (without duplication of amounts theretofore paid by the Issuer), as and when due, regardless of any defense, right of set-off or counterclaim that the Issuer may have or assert other than the defense of payment. The Guarantor’s obligation to make a Guarantee Payment may be satisfied by direct payment of the required amounts by the Guarantor to the Holders or by causing the Issuer to pay such amounts to the Holders.

-11-

Section 5.2. Waiver of Notice and Demand. The Guarantor hereby waives notice of acceptance of this Capital Securities Guarantee and of any liability to which it applies or may apply, presentment, demand for payment, any right to require a proceeding first against the Issuer or any other Person before proceeding against the Guarantor, protest, notice of nonpayment, notice of dishonor, notice of redemption and all other notices and demands.

Section 5.3. Obligations Not Affected. The obligations, covenants, agreements and duties of the Guarantor under this Capital Securities Guarantee shall in no way be affected or impaired by reason of the happening from time to time of any of the following:

(a) the release or waiver, by operation of law or otherwise, of the performance or observance by the Issuer of any express or implied agreement, covenant, term or condition relating to the Capital Securities to be performed or observed by the Issuer;

(b) the extension of time for the payment by the Issuer of all or any portion of the Distributions, Redemption Price, Liquidation Distribution or any other sums payable under the terms of the Capital Securities or the extension of time for the performance of any other obligation under, arising out of, or in connection with, the Capital Securities (other than an extension of time for payment of Distributions, Redemption Price, Liquidation Distribution or other sum payable that results from the extension of any interest payment period on the Debentures permitted by the Indenture);

(c) any failure, omission, delay or lack of diligence on the part of the Holders to enforce, assert or exercise any right, privilege, power or remedy conferred on the Holders pursuant to the terms of the Capital Securities, or any action on the part of the Issuer granting indulgence or extension of any kind;

(d) the voluntary or involuntary liquidation, dissolution, sale of any collateral, receivership, insolvency, bankruptcy, assignment for the benefit of creditors, reorganization, arrangement, composition or readjustment of debt of, or other similar proceedings affecting, the Issuer or any of the assets of the Issuer;

(e) any invalidity of, or defect or deficiency in, the Capital Securities;

(f) the settlement or compromise of any obligation guaranteed hereby or hereby incurred;

(g) the consummation of the Exchange Offer; or

(h) any other circumstance whatsoever that might otherwise constitute a legal or equitable discharge or defense of a guarantor, it being the intent of this Section 5.3 that the obligations of the Guarantor with respect to the Guarantee Payments shall be absolute and unconditional under any and all circumstances.

There shall be no obligation of the Holders to give notice to, or obtain consent of, the Guarantor with respect to the happening of any of the foregoing.

-12-

Section 5.4. Enforcement of Guarantee; Rights of Holders. (a) The Holders of a Majority in liquidation amount of the Capital Securities have the right to direct the time, method and place of conducting any proceeding for any remedy available to the Capital Securities Guarantee Trustee in respect of this Capital Securities Guarantee or exercising any trust or power conferred upon the Capital Securities Guarantee Trustee under this Capital Securities Guarantee; provided, however, that, subject to Section 3.1, the Capital Securities Guarantee Trustee shall have the right to decline to follow any such direction if the Capital Securities Guarantee Trustee shall determine that the action so directed would be unjustly prejudicial to the Holders not taking part in such direction or if the Capital Securities Guarantee Trustee being advised by counsel determines that the action or proceeding so directed may not lawfully be taken or if the Capital Securities Guarantee Trustee shall determine, in good faith by its board of directors or trustees, executive committee, or a trust committee of directors or trustees and/or Responsible Officers, that the action or proceedings so directed would involve the Capital Securities Guarantee Trustee in personal liability.

(b) If the Capital Securities Guarantee Trustee fails to enforce the Capital Securities Guarantee, any Holder may institute a legal proceeding directly against the Guarantor to enforce the Capital Securities Guarantee Trustee’s rights under this Capital Securities Guarantee, without first instituting a legal proceeding against the Issuer, the Capital Securities Guarantee Trustee or any other Person or entity. The Guarantor waives any right or remedy to require that any action be brought first against the Issuer or any other Person or entity before proceeding directly against the Guarantor.

Section 5.5. Guarantee of Payment. This Capital Securities Guarantee creates a guarantee of payment and not of collection.

Section 5.6. Subrogation. The Guarantor shall be subrogated to all, if any, rights of the Holders against the Issuer in respect of any amounts paid to such Holders by the Guarantor under this Capital Securities Guarantee; provided, however, that the Guarantor shall not (except to the extent required by mandatory provisions of law) be entitled to enforce or exercise any right that it may acquire by way of subrogation or any indemnity, reimbursement or other agreement, in all cases as a result of payment under this Capital Securities Guarantee, if, at the time of any such payment, any amounts are due and unpaid under this Capital Securities Guarantee. If any amount shall be paid to the Guarantor in violation of the preceding sentence, the Guarantor agrees to hold such amount in trust for the Holders and to pay over such amount to the Holders.

Section 5.7. Independent Obligations. The Guarantor acknowledges that its obligations hereunder are independent of the obligations of the Issuer with respect to the Capital Securities, and that the Guarantor shall be liable as principal and as debtor hereunder to make Guarantee Payments pursuant to the terms of this Capital Securities Guarantee notwithstanding the occurrence of any event referred to in subsections (a) through (h), inclusive, of Section 5.3 hereof.

-13-

Article VI

Limitation of Transactions; Subordination

Section 6.1. Limitation of Transactions. So long as any Capital Securities remain outstanding, the Guarantor shall not (i) declare or pay any dividends or distributions on, or redeem, purchase, acquire, or make a liquidation payment with respect to, any of the Guarantor’s capital stock (which includes common and preferred stock), (ii) make any payment of principal, interest or premium, if any, on or repay or repurchase or redeem any debt securities issued by the Guarantor (including any Other Debentures) that rank pari passu with or junior in right of payment to the Debentures, or (iii) make any guarantee payments with respect to any guarantee by the Guarantor of any securities of any subsidiary of the Guarantor (including Other Guarantees) if such guarantee ranks pari passu or junior in right of payment to the Debentures (other than (a) dividends or distributions in shares of, or options, warrants, rights to subscribe for or purchase shares of, common stock of the Guarantor, (b) any declaration of a dividend in connection with the implementation of a stockholders’ rights plan, or the issuance of stock under any such plan in the future, or the redemption or repurchase of any such rights pursuant thereto, (c) payments under the Capital Securities Guarantee, (d) as a direct result of, and only to the extent required in order to avoid the issuance of fractional shares of capital stock following, a reclassification of the Guarantor’s capital stock or the exchange or the conversion of one class or series of the Guarantor’s capital stock for another class or series of the Guarantor’s capital stock, (e) the purchase of fractional interests in shares of the Guarantor’s capital stock pursuant to the conversion or exchange provisions of such capital stock or the security being converted or exchanged, and (f) purchases of common stock related to the issuance of common stock or rights under any of the Guarantor’s benefit plans for its directors, officers or employees or the Guarantor’s dividend reinvestment plan) if at such time (i) an Event of Default (as defined in the Indenture) shall have occurred and be continuing, (ii) there shall have occurred any event of which the Guarantor has actual knowledge that (a) is, or with the giving of notice or the lapse of time, or both, would be an Event of Default (as defined in the Indenture), and (b) in respect of which the Guarantor shall not have taken reasonable steps to cure, (iii) if the Debentures are held by the Property Trustee, the Guarantor shall be in default with respect to its payment of any obligations under this Capital Securities Guarantee, or (iv) the Guarantor shall have given notice of its election of the exercise of its right to extend the interest payment period pursuant to Section 16.01 of the Indenture and any such extension shall be continuing.

Section 6.2. Ranking. This Capital Securities Guarantee will constitute an unsecured obligation of the Guarantor and will rank (i) subordinate and junior in right of payment in full of Senior Indebtedness (as defined in the Indenture), to the same extent and in the same manner that the Debentures are subordinated to Senior Indebtedness pursuant to the Indenture, it being understood that the terms of Article XV of the Indenture shall apply to the obligations of the Guarantor under this Capital Securities Guarantee as if (x) such Article XV were set forth herein in full, and (y) such obligations were substituted for the term “Securities” appearing in such Article XV, (ii) pari passu with the Debentures, the Other Debentures and with the most senior preferred or preference stock now or hereafter issued by the Guarantor and with any Other Guarantee (as defined herein) and any guarantee now or hereafter entered into by the Guarantor in respect of any preferred or preference stock of any Affiliate of the Guarantor, and (iii) senior to the Guarantor’s common stock.

-14-

Article VII

Termination

Section 7.1. Termination. This Capital Securities Guarantee shall terminate and be of no further force or effect (i) upon full payment of the Redemption Price (as defined in the Declaration) of all Capital Securities or (ii) upon liquidation of the Trust, the full payment of the amounts payable in accordance with the Declaration or the distribution of the Debentures to the Holders of all of the Capital Securities. Notwithstanding the foregoing, this Capital Securities Guarantee will continue to be effective or will be reinstated, as the case may be, if at any time any Holder must restore payment of any sums paid under the Capital Securities or under this Capital Securities Guarantee.

Article VIII

Compensation and Expenses of Capital Securities Guarantee Trustee

Section 8.1. Compensation and Expenses. The Guarantor covenants and agrees to pay to the Capital Securities Guarantee Trustee from time to time, and the Capital Securities Guarantee Trustee shall be entitled to, such compensation as shall be agreed to in writing between the Guarantor and the Capital Securities Guarantee Trustee (which shall not be limited by any provision of law in regard to the compensation of a trustee of an express trust), and the Guarantor will pay or reimburse the Capital Securities Guarantee Trustee upon its request for all reasonable expenses, disbursements and advances incurred or made by the Capital Securities Guarantee Trustee in accordance with any of the provisions of this Capital Securities Guarantee (including the reasonable compensation and the expenses and disbursements of its counsel and of all Persons not regularly in its employ) except any such expense, disbursement or advance as may arise from or be incurred in connection with its negligence or bad faith. The Guarantor also covenants to indemnify each of the Capital Securities Guarantee Trustee (and its officers, agents, directors and employees) for, and to hold it harmless against, any and all loss, damage, claim, liability or expense including taxes (other than taxes based on the income of the Capital Securities Guarantee Trustee) incurred without negligence or bad faith on the part of the Capital Securities Guarantee Trustee and arising out of or in connection with the acceptance or administration of this guarantee, including the costs and expenses of defending itself against any claim of liability in the premises.

The provisions of this Article shall survive the resignation or removal of the Capital Securities Guarantee Trustee and the termination of this Capital Securities Guarantee.

Article IX

Indemnification

Section 9.1. Exculpation. (a) No Indemnified Person shall be liable, responsible or accountable in damages or otherwise to the Guarantor or any Covered Person (as defined in the Declaration) for any loss, damage or claim incurred by reason of any act or omission performed or omitted by such Indemnified Person in good faith in accordance with this

-15-

Capital Securities Guarantee and in a manner that such Indemnified Person reasonably believed to be within the scope of the authority conferred on such Indemnified Person by this Capital Securities Guarantee or by law, except that an Indemnified Person shall be liable for any such loss, damage or claim incurred by reason of such Indemnified Person’s negligence or willful misconduct with respect to such acts or omissions.

(b) An Indemnified Person shall be fully protected in relying in good faith upon the records of the Guarantor and upon such information, opinions, reports or statements presented to the Guarantor by any Person as to matters the Indemnified Person reasonably believes are within such other Person’s professional or expert competence and, if selected by such Indemnified Person, has been selected by such Indemnified Person with reasonable care, including information, opinions, reports or statements as to the value and amount of the assets, liabilities, profits, losses, or any other facts pertinent to the existence and amount of assets from which Distributions to Holders might properly be paid.

Section 9.2. Indemnification. The Guarantor agrees to indemnify each Indemnified Person for, and to hold each Indemnified Person harmless against, any and all loss, liability, damage, claim or expense incurred without negligence or bad faith on its part, arising out of or in connection with the acceptance or administration of the trust or trusts hereunder, including the costs and expenses (including reasonable legal fees and expenses) of defending itself against, or investigating, any claim or liability in connection with the exercise or performance of any of its powers or duties hereunder. The obligation to indemnify as set forth in this Section 9.2 shall survive the resignation or removal of the Capital Securities Guarantee Trustee and the termination of this Capital Securities Guarantee.

Article X

Miscellaneous

Section 10.1. Successors and Assigns. All guarantees and agreements contained in this Capital Securities Guarantee shall bind the successors, assigns, receivers, trustees and representatives of the Guarantor and shall inure to the benefit of the Holders then outstanding.

Section 10.2. Amendments. Except with respect to any changes that do not materially adversely affect the rights of Holders (in which case no consent of Holders will be required), this Capital Securities Guarantee may only be amended with the prior approval of the Holders of a Majority in liquidation amount of the Capital Securities (including the stated amount that would be paid on redemption, liquidation or otherwise, plus accrued and unpaid Distributions to the date upon which the voting percentages are determined). The provisions of the Declaration with respect to consents to amendments thereof (whether at a meeting or otherwise) shall apply to the giving of such approval.

-16-

Section 10.3. Notices. All notices provided for in this Capital Securities Guarantee shall be in writing, duly signed by the party giving such notice, and shall be delivered, telecopied or mailed by first class mail, as follows:

(a) If given to the Issuer, in care of the Administrative Trustee at the Issuer’s mailing address set forth below (or such other address as the Issuer may give notice of to the Holders and the Capital Securities Guarantee Trustee):

First Midwest Capital Trust I

c/o First Midwest Bancorp, Inc.

One Pierce Place, Suite 1500

Itasca, Illinois 60143

Attention: Corporate Secretary

Telecopy: 630 ###-###-####

(b) If given to the Capital Securities Guarantee Trustee, at the Capital Securities Guarantee Trustee’s mailing address set forth below (or such other address as the Capital Securities Guarantee Trustee may give notice of to the Holders and the Issuer):

Wilmington Trust Company

Rodney Square North

1100 North Market Street

Wilmington, Delaware ###-###-####

Attention: Corporate Trust Administration

Telecopy: 302 ###-###-####

(c) If given to the Guarantor, at the Guarantor’s mailing address set forth below (or such other address as the Guarantor may give notice of to the Holders and the Capital Securities Guarantee Trustee):

First Midwest Bancorp, Inc.

One Pierce Place, Suite 1500

Itasca, Illinois 60143

Attention: Corporate Secretary

Telecopy: 630 ###-###-####

(d) If given to any Holder, at the address set forth on the books and records of the Issuer.

All such notices shall be deemed to have been given when received in person, telecopied with receipt confirmed, or mailed by first class mail, postage prepaid except that if a notice or other document is refused delivery or cannot be delivered because of a changed address of which no notice was given, such notice or other document shall be deemed to have been delivered on the date of such refusal or inability to deliver.

Section 10.4. Benefit. This Capital Securities Guarantee is solely for the benefit of the Holders and, subject to Section 3.1(a), is not separately transferable from the Capital Securities.

Section 10.5. Counterparts. This Capital Securities Guarantee may be signed in any number of counterparts, each of which shall be an original, with the same effect as if the signatures thereto and hereto were upon the same instrument.

-17-

Section 10.6. Governing Law. This Capital Securities Guarantee shall be governed by, and construed and interpreted in accordance with the laws of the State of Illinois, without regard to conflicts of laws principles thereof.

-18-

This Capital Securities Guarantee is executed as of the day and year first above written.

-19-