VOTING AGREEMENT

Exhibit 10.1

VOTING AGREEMENT

This VOTING AGREEMENT, dated as of September 4, 2008 (this “Agreement”), is by and between Taylor Capital Group, Inc., a Delaware corporation (the “Company”) and the stockholders listed on the signature pages hereto under the heading “Stockholders” (each, a “Stockholder,” and collectively, the “Stockholders”).

RECITALS

WHEREAS, the Company and certain investors (each a “Buyer” and collectively, the “Buyers”) have entered into that certain Securities Purchase Agreement, dated as of September 4, 2008 (the “Securities Purchase Agreement”), pursuant to which, among other things, the Company has agreed to issue and sell to the Buyers, and the Buyers have agreed to purchase, shares of Series A Non-Cumulative Convertible Perpetual Preferred Stock of the Company (the “Preferred Shares”), which will, among other things, be convertible into shares of the Company’s common stock, par value $0.01 value per share (the “Common Stock”), in accordance with the terms to be set forth in the Third Amended and Restated Certificate of Incorporation of the Company;

WHEREAS, in the event of a Potential Delay, under certain circumstances set forth in the Securities Purchase Agreement, the parties may have an initial closing with respect to 2,400,000 shares of Series B Convertible Preferred Stock of the Company (the “Designated Preferred”) on or before September 30, 2008, which Designated Preferred would, if issued, be mandatorily exchangeable for 2,400,000 Preferred Shares upon receipt of Stockholder Approval (as defined in the Securities Purchase Agreement), in accordance with the terms and conditions set forth in the Securities Purchase Agreement.;

WHEREAS, as of the date hereof, the Stockholders own collectively 4,815,880 shares of Common Stock, which represent in the aggregate approximately 43.77% of the total issued and outstanding capital stock of the Company;

WHEREAS, as a condition to the willingness of the Buyers to enter into the Securities Purchase Agreement and to consummate the transactions contemplated thereby (collectively, the “Transaction”), the Company has agreed to enter into and enforce the terms hereof and have required that each Stockholders agree, and in order to induce the Buyers to enter into the Securities Purchase Agreement, each Stockholder has agreed, to enter into this Agreement with respect to all of the Common Stock now owned and which may hereafter be acquired by such Stockholder and any other securities, if any, which such Stockholder is currently entitled to vote, or after the date hereof becomes entitled to vote, at any meeting of stockholders of the Company (collectively, the “Owned Shares”); and

WHEREAS, capitalized terms used in this Agreement and not defined herein shall have the meanings ascribed to such terms in the Securities Purchase Agreement.

NOW, THEREFORE, in consideration of the foregoing and the mutual covenants and agreements contained herein, and intending to be legally bound hereby, the parties hereto hereby agree as follows:

ARTICLE I

VOTING AGREEMENT OF THE STOCKHOLDERS

SECTION 1.01. Voting Agreement. Subject to the terms and conditions hereof, each Stockholder hereby agrees that at any meeting of the stockholders of the Company (including any adjournment or postponement thereof), however called, such Stockholder will (y) appear at such a meeting, or at the Stockholder’s option, otherwise cause its Owned Shares to be counted as present there at, for purposes of calculating a quorum, and (z) vote, or cause to be voted, all of its Owned Shares (A) in favor of the Proposals as described in Section 4(a) of the Securities Purchase Agreement, (B) in favor of any other matter to be approved by the stockholders of the Company to facilitate the transactions contemplated by the Transaction Documents, including the Transaction, and (C) against any proposal, action or agreement not contemplated by the Transaction Documents that would reasonably be expected to impede, interfere with, delay or attempt to discourage the transactions contemplated by the Transaction Documents. Each Stockholder acknowledges receipt and review of a copy of the Securities Purchase Agreement and the other Transaction Documents.

SECTION 1.02. Proxy. Each Stockholder hereby revokes any and all previous proxies and powers of attorney granted with respect to its Owned Shares, and no subsequent proxy or power of attorney (except in furtherance of Stockholder’s obligations under Section 1.01 hereof) shall be given or written consent executed (and if given or executed, shall not be effective) by Stockholder with respect thereto so long as this Agreement remains in effect. By entering into this Agreement, each Stockholder hereby grants a proxy appointing Bruce W. Taylor (“Proxy”), with full power of substitution, as such Stockholder’s attorney-in-fact and proxy, for and in such Stockholder’s name, to be counted as present and to vote (including by written consent, if applicable) or otherwise to act on behalf of such Stockholder with respect to its Owned Shares solely with respect to and in furtherance of the matters described in, and in the manner contemplated by, Section 1.01 hereof. The proxy granted by each Stockholder pursuant to this Section 1.02 is, subject to the penultimate sentence of this Section 1.02, irrevocable and is coupled with an interest, in accordance with Section 212(e) of the Delaware General Corporation Law and is granted in order to secure each Stockholder’s performance under this Agreement and also in consideration of the Buyers entering into this Agreement and the Securities Purchase Agreement. If any Stockholder fails for any reason to be counted as present or to vote such Stockholder’s Owned Shares in accordance with the requirements of Section 1.01 above, then such Proxy shall have the right to cause to be present or vote such Stockholder’s Owned Shares in accordance with the provisions of Section 1.01. The Proxy agrees that, in the event any Stockholder shall not have fully complied with the provisions of Section 1.01 hereof with respect to any such meeting of the Company stockholders (including any adjournment or postponement thereof), the Proxy shall in accordance with this Section 1.02 cause all Owned Shares of such Stockholder to be counted as present at such stockholder meeting, for purposes of calculating a quorum, and to cause all Owned Shares of such Stockholder to be voted with respect to and in furtherance of the matters set forth in, and in the manner contemplated by, Section 1.01 hereof. The proxy granted by each Stockholder shall be automatically revoked upon termination of this Agreement in accordance with its terms. Each Stockholder agrees, from the date hereof until the termination of this Agreement, not to attempt to revoke, frustrate the exercise of, or challenge the validity of, the irrevocable proxy granted pursuant to this Section 1.02.

2

ARTICLE II

REPRESENTATIONS AND WARRANTIES OF THE STOCKHOLDERS

Each Stockholder hereby represents and warrants to the Company as follows:

SECTION 2.01. Authority Relative to This Agreement. Such Stockholder has all necessary power and authority to execute and deliver this Agreement, to perform its obligations hereunder and to consummate the transactions contemplated hereby. This Agreement has been duly executed and delivered by such Stockholder and constitutes a legal, valid and binding obligation of such Stockholder, enforceable against such Stockholder in accordance with its terms, except (a) as such enforceability may be limited by general principles of equity or to applicable bankruptcy, insolvency, reorganization, moratorium, liquidation or other similar laws relating to, or affecting generally, the enforcement of applicable creditors’ rights and remedies; (b) where the remedy of specific performance or other forms of equitable relief may be subject to certain equitable defenses and principles and to the discretion of the court before which the proceeding may be brought; and (c) where rights to indemnity and contribution thereunder may be limited by applicable law and public policy.

SECTION 2.02. No Conflict. The execution and delivery of this Agreement by such Stockholder does not, and the performance of this Agreement by such Stockholder shall not, (a) conflict with or violate any federal, state or local law, statute, ordinance, rule, regulation, order, judgment or decree applicable to such Stockholder or by which such Stockholder’s Owned Securities are bound or affected, or (b) result in any breach of or constitute a default (or an event that with notice or lapse of time or both would become a default) under, or give to others any rights of termination, amendment, acceleration or cancellation of, or result in the creation of a lien or encumbrance on any of such Stockholder’s Owned Securities pursuant to any note, bond, mortgage, indenture, contract, agreement, lease, license, permit or other instrument or obligation to which such Stockholder is a party or by which such Stockholder or such Stockholder’s Owned Securities are bound. The execution and delivery of this Agreement by such Stockholder does not, and the performance of this Agreement by such Stockholder shall not, require any consent, approval, authorization or permit of, or filing with or notification to, any governmental entity or other third party by such Stockholder.

SECTION 2.03. Title to the Stock. As of the date hereof, each Stockholder is the owner of the number of shares of Common Stock set forth opposite such Stockholder’s name on Appendix A attached hereto, entitled to vote, without restriction, on all matters brought before holders of capital stock of the Company, which shares of Common Stock represent on the date hereof the percentage of the outstanding stock and voting power of the Company set forth on Appendix A. Such shares of Common Stock are all the securities of the Company owned, either of record or beneficially, by such Stockholder. Other than with respect to up to One Hundred Thousand (100,000) shares of Common Stock which are owned by the Voting Trust U/A/D 11/30/98 (the “Voting Trust”) and may be pledged pursuant to a consent executed in June 2008, all such shares of Common Stock are owned free and clear of all security interests, liens, claims,

3

pledges, options, rights of first refusal, agreements, limitations on such Stockholder’s voting rights, charges and other encumbrances of any nature whatsoever. Except as provided for herein, no Stockholder has appointed or granted any proxy, which appointment or grant is still effective, with respect to such Stockholder’s Owned Securities.

ARTICLE III

COVENANTS

SECTION 3.01. No Disposition or Encumbrance of Stock. The Stockholders agree not to, and acknowledge that they are not otherwise permitted to for so long as this Agreement is in effect, (a) sell, transfer, tender, assign, hypothecate or otherwise dispose of, grant a proxy or power of attorney with respect to, or create or permit to exist any security interest, lien, claim, pledge, option, right of first refusal, agreement, limitation on such Stockholder’s voting rights, charge or other encumbrance of any nature whatsoever (“Encumbrance”) (other than the Encumbrance with respect to up to 100,000 shares of Common Stock described in Section 2.03 above), in all capacities that such Stockholder shall hold voting and/or dispositive and/or other rights over the Stockholder’s Owned Shares (which capacities, in each case, are sufficient to assure compliance with this Section 3.01), or (b) directly or indirectly, initiate, solicit or encourage any person to take actions which could reasonably be expected to lead to the occurrence of any of the foregoing; provided, however, that any Stockholder may assign, sell or transfer any Common Stock if any recipient of the Common Stock has delivered to the Company a written agreement in a form reasonably satisfactory to the Company that such recipient shall be bound by, and the Common Stock so transferred, assigned or sold shall remain subject to this Agreement; provided, further, however, that the provisions of this Section 3.01 shall apply only to (i) the Cindy L. Taylor Gift Trust U/A/D 6/10/82, the Bruce Taylor Gift Trust U/A/D 6/10/82, the Jeffrey W. Taylor Gift Trust U/A/D 6/10/82 and Bruce Taylor and (ii) those depositors to the Voting Trust that are listed on Appendix B (including all of the Owned Shares set forth next to such depositor’s name on Appendix B), and shall not apply to those depositors to the Voting Trust that are listed on Appendix C (or the Owned Shares set forth next to such depositor’s name on Appendix C); provided, still further, however, the Trustees of the Voting Trust shall promptly notify the Buyers in the event that any action is proposed to be taken by any depositor listed on Appendix C which shall cause the Voting Trust to be prevented from voting the Owned Shares set forth next to such depositor’s name on Appendix C in accordance with the terms of Article I hereof.

SECTION 3.02. Company Cooperation. The Company hereby covenants and agrees that it will not, and each Stockholder irrevocably and unconditionally acknowledges and agrees that the Company will not (and waives any rights against the Company in relation thereto), recognize any Encumbrance or agreement on any of the Common Stock subject to this Agreement unless the provisions of Section 3.01 hereof have been complied with. The Company further covenants and agrees that it shall not give effect to any vote cast or other action taken by any Stockholder with respect to any matter submitted to a vote of the stockholders of the Company, unless such vote or action is in accordance with the terms of this Agreement.

4

ARTICLE IV

TERMINATION

SECTION 4.01. Termination. This Agreement shall terminate without further action (a) upon termination of the Securities Purchase Agreement in accordance with the terms and conditions thereof, (b) with the written consent of the parties hereto, or (c) upon receipt of the Stockholder Approval.

SECTION 4.02. Effect of Termination. Upon termination of this Agreement, the rights and obligations of all the parties will terminate and become void without further action by any party. For the avoidance of doubt, the termination of this Agreement shall not relieve any party of liability for any breach of this Agreement prior to the time of termination.

ARTICLE V

MISCELLANEOUS

SECTION 5.01. Action in Stockholder Capacity Only. The parties acknowledge that this Agreement is entered into by the Stockholders in each Stockholder’s capacity as the legal owner of such Stockholder’s Owned Shares and nothing in this Agreement restricts or limits any action taken by any Stockholder, or trustee or investment advisor of any Stockholder, solely in his or her capacity as a director or officer of the Company and the taking of any actions (or failure to act) solely in his or her capacity as an officer or director of the Company will not be deemed to constitute a breach of this Agreement.

SECTION 5.02. Further Assurances. Each Stockholder will execute and deliver such further documents and instruments and take all further action as may be reasonably necessary in order to consummate the transactions contemplated hereby.

SECTION 5.03. Parties in Interest. Other than with respect to the parties to this Agreement and as provided in the Securities Purchase Agreement, nothing in this Agreement, express or implied, is intended to or shall confer upon any person any right, benefit or remedy of any nature whatsoever under or by reason of this Agreement.

SECTION 5.04. Specific Performance. The parties hereto agree that irreparable damage would occur in the event any provision of this Agreement was not performed in accordance with the terms hereof and that the Company shall be entitled to specific performance of the terms hereof, in addition to any other remedy at law or in equity.

SECTION 5.05. Entire Agreement. This Agreement constitutes the entire agreement among the parties hereto with respect to the subject matter hereof and supersedes all prior agreements and understandings, both written and oral, among the parties with respect to the subject matter hereof.

SECTION 5.06. Amendment. This Agreement may not be amended except by an instrument in writing signed by each of the parties hereto.

5

SECTION 5.07. Severability. If any term or other provision of this Agreement is invalid, illegal or incapable of being enforced by any rule of law, or public policy, all other conditions and provisions of this Agreement shall nevertheless remain in full force and effect so long as the economic or legal substance of this Agreement is not affected in any manner materially adverse to any party. Upon such determination that any term or other provision is invalid, illegal or incapable of being enforced, the parties hereto shall negotiate in good faith to modify this Agreement so as to effect the original intent of the parties as closely as possible in a mutually acceptable manner in order that the terms of this Agreement remain as originally contemplated to the fullest extent possible.

SECTION 5.08. Governing Law; Jurisdiction; Jury Trial. All questions concerning the construction, validity, enforcement and interpretation of this Agreement shall be governed by the internal laws of the State of Delaware, without giving effect to any choice of law or conflict of law provision or rule (whether of the State of Delaware or any other jurisdictions) that would cause the application of the laws of any jurisdictions other than the State of Delaware. Each party hereby irrevocably submits to the exclusive jurisdiction of the state and federal courts sitting in the State of Delaware for the adjudication of any dispute hereunder or in connection herewith or with any transaction contemplated hereby or discussed herein, and hereby irrevocably waives, and agrees not to assert in any suit, action or proceeding, any claim that it is not personally subject to the jurisdiction of any such court, that such suit, action or proceeding is brought in an inconvenient forum or that the venue of such suit, action or proceeding is improper. Each party hereby irrevocably waives personal service of process and consents to process being served in any such suit, action or proceeding by mailing a copy thereof to such party at the address for such notices to it under this Agreement and agrees that such service shall constitute good and sufficient service of process and notice thereof. Nothing contained herein shall be deemed to limit in any way any right to serve process in any manner permitted by law. EACH PARTY HEREBY IRREVOCABLY WAIVES ANY RIGHT IT MAY HAVE, AND AGREES NOT TO REQUEST, A JURY TRIAL FOR THE ADJUDICATION OF ANY DISPUTE HEREUNDER OR IN CONNECTION WITH OR ARISING OUT OF THIS AGREEMENT OR ANY TRANSACTION CONTEMPLATED HEREBY.

SECTION 5.09. Counterparts; Effectiveness. This Agreement may be executed in two or more identical counterparts, all of which shall be considered one and the same agreement and shall become effective when counterparts have been signed by each party and delivered to each other party. In the event that any signature to this Agreement or any amendment hereto is delivered by facsimile transmission or by e-mail delivery of a “.pdf” format data file, such signature shall create a valid and binding obligation of the party executing (or on whose behalf such signature is executed) with the same force and effect as if such facsimile or “.pdf” signature page were an original thereof. No party hereto shall raise the use of a facsimile machine or e-mail delivery of a “.pdf” format data file to deliver a signature to this Agreement or any amendment hereto or the fact that such signature was transmitted or communicated through the use of a facsimile machine or e-mail delivery of a “.pdf” format data file as a defense to the formation or enforceability of a contract and each party hereto forever waives any such defense.

[Signature Pages Follow]

6

IN WITNESS WHEREOF, the parties hereto have duly executed this Agreement on the date first written above.