VOTING AGREEMENT

EXHIBIT 10.1.40

VOTING AGREEMENT

     THIS VOTING AGREEMENT (this “Agreement”), dated as of November ___, 2004, is entered into Specialty Underwriters’ Alliance, Inc., a Delaware corporation (the “Company”) and Friedman, Billings, Ramsey Group, Inc., a Virginia corporation (“FBR Group”).

W I T N E S S E T H:

     WHEREAS, FBR Group desires to enter this voting agreement with the Company with respect to any and all shares of Company Common Stock (as defined below) held by FBR Group in excess of five percent (5%) of the outstanding shares of Company Common Stock.

     NOW, THEREFORE, in consideration of the foregoing and the mutual representations, warranties, covenants and agreements contained herein, the parties hereto, intending to be legally bound, hereby agree as follows:

     Section 1. Definitions. For purposes of this Agreement:

     “Affiliate” has the meaning specified in Rule 13e-3 promulgated under the Securities Exchange Act of 1934, as amended.

     “Company” has the meaning shown above.

     “Company Common Stock” means the Company’s common stock, par value $0.01 per share.

     “Expiration Date” means the earlier of (a) the date on which FBR Group ceases to constructively or beneficially own any shares of Company Common Stock and (b) a date not earlier than the second anniversary of the date of this Agreement so specified by FBR Group by at least 75 days advance notice to the Company; provided that, in respect of clause (b) and notwithstanding any notice given by FBR Group, this Agreement shall not terminate on the proposed Expiration Date if the effect of such termination would be to cause the Company to be an Affiliate of FBR Group as of such proposed date..

     “FBR Group” has the meaning shown above.

     “FBR Group Shares” means the ___shares of Company Common Stock owned by FBR Group on the date hereof, (b) any shares of Company Common Stock distributed to FBR Group prior to the Expiration Date by reason of a stock dividend, split-up, recapitalization, reclassification, combination, merger, exchange of shares or otherwise and (c) any other shares of Company Common Stock of which FBR Group has “beneficial ownership” (as defined in Rule 13d-3 promulgated under the Securities Exchange Act of 1934, as amended) prior to the Expiration Date.

     Section 2. Agreement to Vote Shares. Until the Expiration Date, FBR Group shall, at any meeting of the holders of Company Common Stock, however such meeting is called and regardless of whether such meeting is a special or annual meeting of stockholders of the

Company, or at any adjournment thereof, or in connection with any written consent of stockholders of the Company, vote, or cause to be voted, the FBR Group Shares in excess of five percent (5%) of the outstanding shares of Company Common Stock in the same proportion that the Company’s other stockholders vote their shares of Company Common Stock with respect to any proposal submitted to the Company’s stockholders for a vote, so that, as a result, the percentage of the FBR Group Shares in excess of five percent (5%) of the outstanding shares of Company Common Stock that are voted in favor of such proposal shall equal the percentage of the outstanding shares of Company Common Stock held by all other Company stockholders voted in favor of such proposal and the percentage of the FBR Group Shares in excess of five percent (5%) of the outstanding shares of Company Common Stock that are voted against such proposal equals the percentage of the outstanding shares of Company Common Stock held by all other Company stockholders voted against such proposal.

     Section 3. Covenants of FBR Group. FBR Group hereby agrees and covenants that, during the period from the date of this Agreement through the Expiration Date,

     (a) Restrictions on Proxies and Voting Arrangements. FBR Group shall not (i) grant any proxy, power-of-attorney or other authorization in or with respect to the FBR Group Shares inconsistent with the provisions of Section 2 or (ii) deposit any of the FBR Group Shares into a voting trust or enter into a voting agreement or arrangement with respect to any of the FBR Group Shares.

     (b) No Inconsistent Arrangements. FBR Group shall not take any other action that would in any material way restrict, limit or interfere with the performance of any of FBR Group’s obligations hereunder.

     Section 4. Representations and Warranties.

     (a) FBR Group hereby represents and warrants to the Company as follows:

         (i) No Inconsistent Proxies, Voting Trusts, Etc. FBR Group has not (i) granted any proxy, power-of-attorney or other authorization in or with respect to the FBR Group Shares or (ii) deposited any of the FBR Group Shares into a voting trust or entered into a voting agreement or arrangement with respect to any of the FBR Group Shares.

         (ii) Power; Binding Agreement. FBR Group has the power (or, if applicable, corporate power) and authority to enter into and perform all of its obligations hereunder, including, without limitation, the power and authority to vote the FBR Group Shares in accordance with Section 2 hereof. The execution, delivery and performance of this Agreement by FBR Group will not violate any other agreement to which FBR Group is a party including, without limitation, any voting agreement, proxy arrangement, pledge agreement, stockholders agreement, voting trust or trust agreement. This Agreement has been duly and validly executed and delivered by FBR Group, and constitutes a valid and binding agreement of FBR Group, enforceable against the FBR Group in accordance with its terms, except that (A) such enforcement may be subject to applicable bankruptcy, insolvency, moratorium, or other similar laws, now or hereafter in effect, affecting creditors’ rights generally and (B) the remedy of specific performance and injunctive and other forms of equitable relief may be subject to

2

equitable defenses and to the discretion of the court before which any proceeding therefor may be brought. There is no beneficiary or holder of a voting trust certificate or other interest of any trust of which FBR Group is a trustee whose consent is required for the execution and delivery of this Agreement or the compliance by FBR Group with the terms hereof, other than such consents as have been obtained.

         (iii) No Conflicts. None of the execution and delivery of this Agreement by FBR Group, the consummation by FBR Group of the transactions contemplated hereby or compliance by FBR Group with any of the provisions hereof shall (A) conflict with, or result in any breach of, any organizational documents applicable to FBR Group, (B) result in a violation or breach of, or constitute (with or without due notice or lapse of time or both) a default (or give rise to any third party right of termination, cancellation, material modification or acceleration) under any of the terms, conditions or provisions of any note, loan agreement, bond, mortgage, indenture, license, contract, commitment, arrangement, understanding, agreement or other instrument or obligation of any kind to which FBR Group is a party or by which FBR Group or any of the FBR Group Shares may be bound, or (C) violate any order, writ, injunction, decree, judgment, order, statute, arbitration award, rule or regulation applicable to FBR Group or any of the FBR Group Shares, in each case, in a manner that could reasonably be expected to materially hinder or impede FBR Group’s ability to perform their obligations hereunder.

     (b) The Company represents and warrants to FBR Group as follows:

         (i) Power; Binding Agreement. The Company has the corporate power and authority to enter into and perform all of its obligations under this Agreement. This Agreement has been duly and validly executed and delivered by the Company and constitutes a valid and binding agreement of the Company, enforceable in accordance with its terms, except that (A) such enforcement may be subject to applicable bankruptcy, insolvency, moratorium, or other similar laws, now or hereafter in effect, affecting creditors’ rights generally and (B) the remedy of specific performance and injunctive and other forms of equitable relief may be subject to equitable defenses and to the discretion of the court before which any proceeding therefor may be brought..

     Section 5. Termination. This Agreement and the covenants, representations, warranties, and agreements contained herein shall terminate upon the Expiration Date. Upon any termination of this Agreement, this Agreement shall thereupon become void and of no further force and effect, and there shall be no liability in respect of this Agreement on the part of any party hereto or any of its directors, officers, partners, stockholders, employees, agents, advisors, representatives or Affiliates; provided, however, that nothing contained herein shall relieve any party from any liability for such party’s willful breach of this Agreement prior to termination.

     Section 6. Miscellaneous.

     (a) Notices. All notices, requests, claims, demands and other communications hereunder shall be in writing and shall be given or made (and shall be deemed to have been duly given or made upon receipt) by delivery in person, by facsimile (with a confirmatory copy sent by overnight courier), by overnight courier service or by registered or certified mail (postage prepaid, return receipt requested) to the respective parties at the following addresses (or at such

3

other address for a party as shall be specified in a notice given in accordance with this Section 6(a)):

               if to the Company:

               Specialty Underwriters’ Alliance, Inc.
               8585 Stemmons Freeway., Suite 600
               Suite 200, South Tower
               Dallas, Texas 75247
               Attention: Secretary
               Fax: 214 ###-###-####

               if to FBR Group:

               Friedman, Billings, Ramsey Group, Inc.
               1001 19th Street North
               Arlington, Virginia 22209
               Attention: General Counsel
               Fax: 703 ###-###-####

     (b) Severability. If any term or other provision of this Agreement is invalid, illegal or incapable of being enforced by any rule of law, or public policy, all other conditions and provisions of this Agreement shall nevertheless remain in full force and effect. Upon such determination that any term or other provision is invalid, illegal or incapable of being enforced, the parties hereto shall negotiate in good faith to modify this Agreement so as to effect the original intent of the parties as closely as possible.

     (c) Entire Agreement. This Agreement constitutes the entire agreement among the parties with respect to the subject matter hereof and supersedes all prior agreements and undertakings, both written and oral, among the parties, or any of them, with respect to the subject matter hereof.

     (d) Assignment. The provisions of this Agreement shall be binding upon and inure to the benefit of the parties hereto and their respective successors and assigns, provided that, except as provided herein, no party may assign, delegate or otherwise transfer any of its rights or obligations hereunder, in whole or in part, by operation of law or otherwise by any of the parties, without the consent of the other party hereto.

     (e) Parties in Interest. This Agreement shall be binding upon and inure solely to the benefit of each party hereto, and nothing herein, express or implied, is intended to or shall confer upon any other Person any right, benefit or remedy of any nature whatsoever under or by reason of this Agreement.

     (f) Specific Performance. The parties hereto agree that irreparable damage would occur in the event any provision of this Agreement was not performed in accordance with the terms hereof and that the parties shall be entitled to specific performance of the terms hereof, in addition to any other remedy at law or equity.

4

     (g) Governing Law. This Agreement shall be governed by, and construed in accordance with, the laws of the State of Delaware, regardless of the laws that might otherwise govern under applicable principles of conflicts of law) as to all matters, including, but not limited to, matters of validity, construction, effect, performance and remedies.

     (h) Descriptive Headings; Interpretation. The descriptive headings herein are inserted for convenience of reference only and are not intended to be part of or to affect the meaning or interpretation of this Agreement.

     (i) Amendment, Modification and Waiver. This Agreement may not be amended, modified or waived except by an instrument or instruments in writing signed and delivered on behalf of the party hereto against whom such amendment, modification or waiver is sought to be entered.

     (j) Counterparts. This Agreement may be executed and delivered (including by facsimile transmission) in one or more counterparts, and by the different parties hereto in separate counterparts, each of which when executed and delivered shall be deemed to be an original but all of which taken together shall constitute one and the same agreement.

5

     IN WITNESS WHEREOF, the Company and FBR Group have caused this Agreement to be duly executed as of the day and year first above written.

6