ASSIGNMENT AND ASSUMPTION AGREEMENT

Exhibit 10.4

FORM OF ASSIGNMENT AND ASSUMPTION AGREEMENT

ASSIGNMENT AND ASSUMPTION AGREEMENT

THIS ASSIGNMENT AND ASSUMPTION AGREEMENT (this “Agreement”) is dated as of May __, 2009, and is by and between FBR CAPITAL MARKETS CORPORATION, a corporation organized under the laws of the Commonwealth of Virginia (“CMC”), and FRIEDMAN, BILLINGS, RAMSEY GROUP, INC. (d/b/a ARLINGTON ASSET INVESTMENT CORP.), a corporation organized under the laws of the Commonwealth of Virginia (“Group”). CMC and Group are sometimes referred to herein individually as a “Party” and collectively as the “Parties.”

RECITALS

WHEREAS, pursuant to that certain Stock Repurchase Agreement, dated as of May __, 2009, by and between CMC, FBR TRS Holdings, Inc. (“FBR TRS”), and Group (the “Repurchase Agreement”), CMC has agreed to repurchase certain shares of capital stock of CMC that are currently held of record by FBR TRS and beneficially owned by Group; and

WHEREAS, in connection with the entering into of the Repurchase Agreement, Group desires to assign to CMC, and CMC desires to accept and receive from Group, all of Group’s contract rights under certain agreements to which it is a party all upon the terms and subject to the conditions set forth in this Agreement.

NOW, THEREFORE, in consideration of the premises and the mutual covenants and agreements set forth herein, and for other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the Parties hereby agree as follows:

AGREEMENT

Section 1. Assignment and Assumption. Group hereby assigns, sells, transfers and conveys each applicable Assigned Contract (as hereinafter defined) (with such assignment, sale, transfer and conveyance deemed to have been effective as of the original date of each applicable Assigned Contract) (the “Assignment”) to CMC and all of the rights, title and interest of Group in and to, and all of Group’s obligations in connection with, each of the agreements identified on Exhibit A hereto (together with any agreements identified in accordance with the terms of Section 2, the “Assigned Contracts”). CMC hereby accepts the Assignment and assumes and agrees (with such assumption and agreement deemed to have been effective as of the original date of each applicable Assigned Contract) to perform all of the duties and obligations of Group under the Assigned Contracts; provided, however, that notwithstanding the foregoing, Group shall retain all responsibility for, and CMC shall have no obligation or liabilities with respect to, any such contracts to the extent that such contracts pertain solely to Group separate and apart from CMC or any subsidiary thereof.

Section 2. Additional Agreements. The Parties agree that in the event that Group or CMC later identifies any agreement to which Group is a party that the Parties mutually agree in good faith relates at such time, previously related, or will relate primarily to CMC’s business,

Group will assign such agreement to CMC and CMC will assume Group’s duties and obligations thereunder, subject to the proviso in Section 1.

Section 3. Approvals and Consents; Further Actions.

(a) In the event an Assigned Contract and any claim, right, benefit, duty or obligation arising thereunder or resulting therefrom is not assignable or assumable without the consent of the other party(ies) thereto, the Parties will use commercially reasonable efforts to obtain as expeditiously as possible the written consent of the other party(ies) to such contract for the assignment or, if required, novation thereof to CMC, and will execute and deliver any other documents and instruments as are reasonably necessary in furtherance of such assignment or novation. The Parties agree to equally share all out-of-pocket costs and expenses (but not internal or employee costs or expenses or allocations therefor) incurred in connection with this Section 3(a). In furtherance of the foregoing, as soon as practicable following the date hereof, CMC shall submit to the other party or parties to any such contract documentation seeking the written waiver or approval of such other contracting party or parties thereto to the transfer, assignment, novation, or amendment of all of the applicable Group claims, rights, benefits, and liabilities thereunder to CMC. In no event shall Group or CMC be obligated to pay any money (other than a de minimis amount) to any person or entity or to offer or grant other financial or other accommodations to any person or entity in connection with obtaining any consent, waiver, confirmation, novation, amendment, or approval with respect to any Assigned Contract.

(b) Until such time as any required consent, waiver, confirmation, novation, or approval is obtained with respect to any Assigned Contract, the Parties will cooperate diligently and in good faith to establish a lawful and commercially reasonable arrangement under which CMC would obtain, to the fullest extent practicable and not prohibited by applicable law, the economic claims, rights, and benefits and assume the corresponding economic burdens and obligations thereunder in accordance with this Agreement (including by means of any subcontracting, sublicensing, or subleasing arrangement) or under which Group would enforce such Assigned Contract at the direction of and for the benefit of CMC, provided, that all reasonable out-of-pocket expenses of such cooperation and related actions shall be paid by CMC. In such event (i) Group will promptly pay, assign, and remit to CMC when received all monies and other consideration received by it under any such Assigned Contract or any claim, right, or benefit arising thereunder, and (ii) CMC will promptly pay, perform, or discharge when due any obligation or liability arising thereunder.

(c) Notwithstanding anything in this Agreement to the contrary, this Agreement shall not constitute an agreement to assign or assume any Assigned Contract or any claim, right, benefit, duty or obligation arising thereunder or resulting therefrom if an attempted assignment or assumption thereof, without the consent of a third party thereto, would constitute a breach or other contravention thereof, be ineffective with respect to any party thereto or in any way adversely affect the rights of Group or CMC thereunder.

(d) Each of the Parties covenants and agrees to execute and deliver, at the request of the other Party, such further instruments of transfer and assignment and to take such

2

other action as such other Party may reasonably request to more effectively consummate the assignment and assumption of the Assigned Contracts as contemplated by this Agreement.

Section 4. Notices. All notices to be given or otherwise made to any Party to this Agreement shall be delivered in accordance with the terms set forth in the Repurchase Agreement.

Section 5. Successors and Assigns. This Agreement shall be binding upon, and shall inure to the benefit of, the Parties and their respective successors and permitted assigns.

Section 6. Governing Law; Jurisdiction. This Agreement shall be governed by and construed in accordance with the internal laws of the Commonwealth of Virginia, without giving effect to any choice of law or conflict of law provisions or rule (whether of the Commonwealth of Virginia or any other jurisdiction) that would cause the application of the laws of any other jurisdiction other than the Commonwealth of Virginia. The Parties agree, for the purposes of any action arising out of or relating to this Agreement, to commence any such action solely in the state or federal courts located in the Commonwealth of Virginia.

Section 7. WAIVER OF JURY TRIAL. EACH OF THE PARTIES HEREBY IRREVOCABLY WAIVES ANY AND ALL RIGHT TO TRIAL BY JURY IN ANY DISPUTE ARISING UNDER OR IN CONNECTION WITH THIS AGREEMENT OR THE TRANSACTIONS CONTEMPLATED HEREBY.

Section 8. Amendments; Modifications. This Agreement may not be amended, supplemented or modified except by an agreement in writing that makes specific reference to this Agreement and which is signed by an authorized representative of each of the Parties. The failure of a Party to assert any of its rights hereunder shall not constitute a waiver of such rights nor in any way affect the validity of this Agreement or any part hereof or the right of such Party thereafter to enforce each and every provision of this Agreement. No waiver of any breach of or noncompliance with this Agreement shall be held to be a waiver of any other or subsequent breach or noncompliance.

Section 9. Entire Agreement. This Agreement and all exhibits hereto constitutes the entire agreement of the Parties with respect to the subject matter hereof and supersedes all prior agreements and understandings, both written and oral, between the Parties with respect to the subject matter hereof.

Section 10. Specific Performance. The Parties acknowledge and agree that irreparable damage would occur if any of the provisions of this Agreement were not performed in accordance with their specific terms or were otherwise breached. Accordingly, each of the Parties agrees that, in the event of any breach of the provisions of this Agreement, the non-breaching Party, without prejudice to any rights to judicial relief it may otherwise have, shall be entitled to seek equitable relief, including injunction, and to enforce specifically the terms and provisions of this Agreement. Each of the Parties (to the extent such Party is the breaching Party) further agrees that it will not oppose the granting of such relief on the basis that the non-breaching Party has an adequate remedy at law.

3

Section 11. Severability. Wherever possible, each provision hereof shall be interpreted in such a manner as to be effective and valid under applicable law. In case any one or more of the provisions contained herein shall, for any reason, be held to be invalid, illegal or unenforceable in any respect, such provision or provisions shall be ineffective to the extent, but only to the extent, of such invalidity, illegality or unenforceability, and the remainder of the provisions of this Agreement shall remain in full force and effect and shall in no way be affected, impaired or invalidated so long as the economic or legal substance of the transactions contemplated hereunder are not affected in any manner materially adverse to any Party. Upon such a determination, the Parties shall negotiate in good faith to modify this Agreement so as to effect the original intent of the Parties as closely as possible in a reasonably acceptable manner so that the transactions contemplated hereunder may be consummated as originally contemplated to the fullest extent possible.

Section 12. No Third-Party Beneficiaries. Nothing contained in this Agreement or in any instrument or document executed by any Party in connection with the transactions contemplated hereby shall create any rights in, or be deemed to have been executed for the benefit of, any Person that is not a Party or a successor or permitted assign of such a Party.

Section 13. Interpretation; Construction. For purposes of this Agreement: (a) the words “include,” “includes” and “including” shall be deemed to be followed by the words “without limitation”; (b) the word “or” is not exclusive; and (c) the words “herein,” “hereof,” “hereby,” “hereto” and “hereunder” refer to this Agreement as a whole. Unless the context otherwise requires, references herein (i) to Sections mean the Sections of this Agreement and (ii) to an agreement, instrument or other document means such agreement, instrument or other document as amended, supplemented and modified from time to time to the extent permitted by the provisions thereof and by this Agreement. Titles and headings to articles or sections herein are inserted for the convenience of reference only and are not intended to be part of or to affect the meaning or interpretation of this Agreement. Ambiguities, inconsistencies or conflicts in this Agreement will not be strictly construed against either Party but will be resolved by applying the most reasonable interpretation under the circumstances, giving full consideration to the Parties’ intentions at the time this Agreement is entered into and common practice in the industry. This Agreement shall be construed as if drafted jointly by all of the Parties.

Section 14. Counterparts. This Agreement may be executed in counterparts, each of which shall be deemed to be an original, but both of which together shall constitute one and the same instrument. This Agreement shall become binding when both counterparts have been signed by the Parties and delivered to each of the Parties by electronic means or otherwise.

[Signature Page Follows]

4

IN WITNESS WHEREOF, the Parties have caused this Agreement to be executed as of the day and year first above written.

Signature Page to Assignment and Assumption Agreement