ESCROW AGREEMENT

Exhibit 10.54

EXECUTION COPY

ESCROW AGREEMENT

THIS ESCROW AGREEMENT (this “Agreement”), dated as of February 18, 2009, is by and among NOVASTAR MORTGAGE, INC., a Virginia corporation (“NMI”); NOVASTAR FINANCIAL, INC., a Maryland corporation (“NFI”); TABERNA PREFERRED FUNDING I, LTD. (“Taberna I”); TABERNA PREFERRED FUNDING II, LTD. (“Taberna II”); KODIAK CDO I, LTD. (“Kodiak” and, collectively with Taberna I and Taberna II, the “Holders”), and WOLFBLOCK LLP, a Pennsylvania limited liability partnership (“Escrow Agent”).

WITNESETH:

WHEREAS, on the date hereof, NMI, NFI, the Holders and other parties thereto have executed an Exchange Agreement (the “Exchange Agreement”), a fully executed copy of which is attached hereto as Exhibit A, which Exchange Agreement provides for the exchange of certain Securities and Existing Securities (as defined therein) on the terms and conditions set forth therein; and

WHEREAS, on the date hereof, NMI, NFI, the Holders, and other parties thereto have executed a Settlement Agreement (the “Settlement Agreement”), a fully executed copy of which is attached hereto as Exhibit B, which Settlement Agreement provides for the settlement of outstanding claims and actions on the terms and conditions set forth therein; and

WHEREAS, the Settlement Agreement provides for certain settlement amounts, as more particularly set forth therein, to be deposited with Escrow Agent, which settlement amounts shall be held and disposed of by Escrow Agent in accordance with the terms of this Agreement;

NOW, THEREFORE, in consideration of the premises and the covenants herein contained, the parties hereto hereby agree as follows:

1. Definitions. All terms used herein and not otherwise defined herein have the same meaning that such terms have in the Exchange Agreement or the Settlement Agreement, as applicable.

2. Escrow Agent. Escrow Agent is hereby appointed as escrow agent to hold and distribute the Deposits defined in Section 3 hereof, in accordance with the terms hereof, and Escrow Agent hereby accepts such appointment and agrees to act in such capacity.

3. Deposits. On or about the date hereof, Escrow Agent shall have received the following from NMI:

(a) A wire transfer in the amount of $2,005,738.51 (the “Kodiak Settlement Amount”);

(b) A wire transfer in the amount of $3,316,921.08 (the “Taberna Settlement Amount”); and

(c) A wire transfer in the amount of $387,000 (the “Restructuring Costs Amount”).

Items (a) through (c) referenced above shall, hereinafter, collectively be referred to as the “Deposits”.

Escrow Agent shall invest the Deposits in non-interest bearing accounts, with or through a commercial bank or similar institution having assets on deposit of at least One Billion Dollars. Escrow Agent shall advise the parties hereto of the name and address of such institution and the account number assigned to each of the Deposits.

4. Release Conditions. Escrow Agent’s release and disbursement of the Deposits pursuant to Section 5 below is conditioned upon Escrow Agent’s receipt, on or before 5:00 p.m. (New York time) April 30, 2009 (the “Expiry Date”) of the following (collectively, the “Release Conditions”):

(a) The Deposits;

(b) A copy of the final, non-appealable dismissal order of the Bankruptcy Court dismissing Case No. 08-12125, as issued by the United States Bankruptcy Court for the District of Delaware (the “Dismissal Order”); and

(c) A certificate, substantially in the form of Exhibit C attached hereto, executed by NMI, NFI, and the Holders, certifying that: (i) the exchange of those certain Securities and Existing Securities (as defined in such Exchange Agreement) has occurred in accordance with the terms and conditions of the Exchange Agreement; (ii) those conditions set forth in Section 7 of the Settlement Agreement have been satisfied or waived by the applicable parties thereto; and (iii) Escrow Agent is authorized to disburse the Deposits in accordance with the provisions of this Agreement (the “Certificate”).

5. Closing and Release of Deposits. On the date all of the foregoing Release Conditions have been fully met, Escrow Agent shall:

(a) Disburse the Kodiak Settlement Amount to The Bank of New York Mellon Trust Company, National Association (“BNYM”), as the Kodiak Indenture Trustee (as defined in the Settlement Agreement), in accordance with wire instructions provided by the Kodiak Indenture Trustee to the Escrow Agent with respect to such Kodiak Settlement Amount;

(b) Disburse the Taberna Settlement Amount to BNYM, as the Taberna Indenture Trustee (as defined in the Settlement Agreement), in accordance with wire instructions provided by the Taberna Indenture Trustee to the Escrow Agent with respect to such Taberna Settlement Amount;

2

(c) Disburse $150,000.00 of the Restructuring Costs Amount to Winston & Strawn LLP, as payment for legal fees incurred, in accordance with wire instructions provided to the Escrow Agent;

(d) Disburse $30,000.00 of the Restructuring Costs Amount to Gardere Wynne Sewell LLP, as payment for legal fees incurred, in accordance with wire instructions provided to the Escrow Agent;

(e) Disburse $175,000.00 of the Restructuring Costs Amount to WolfBlock LLP, as payment for legal fees incurred;

(f) Disburse $7,000.00 of the Restructuring Costs Amount to Richards, Layton & Finger, P.A., as payment for legal fees incurred, in accordance with wire instructions provided to the Escrow Agent; and

(g) Disburse the remaining $25,000 of the Restructuring Costs Amount as instructed, such instructions to be in writing and approved by both NMI and the Holders, to pay (or reimburse) expenses of the Trustees (as defined in the Settlement Agreement) and the trustees under the Operative Documents (as defined in the Exchange Agreement), including, to the extent not otherwise provided for in Section 5(d) and (f) hereof, (i) the legal fees of the Trustees and the trustees under the Operative Documents, (ii) expenses to make the 2009 II/B Preferred Securities (as defined in the Exchange Agreement) eligible for clearance and settlement as book-entry securities through the facilities of The Depository Trust Company and listed for trading through the PORTAL Market and (iii) CUSIP fees.

6. Failure of Release Conditions by the Expiry Date. If for any reason, by 5:00 p.m. (New York time) on the Expiry Date, the foregoing Release Conditions have not been fully met, Escrow Agent shall immediately return the Kodiak Settlement Amount, the Taberna Settlement Amount and the Restructuring Costs Amount to NMI, pursuant to wire instructions provided by NMI to Escrow Agent.

7. Termination. This Agreement shall terminate upon the earlier to occur of (a) the disbursement of the Deposits pursuant to Section 5 or (b) the return of the Deposits pursuant to Section 6. Upon such termination, Escrow Agent shall thereupon and thereafter be free and discharged of all obligations and liabilities hereunder.

8. Escrow Agent.

(a) As escrow agent hereunder, Escrow Agent shall have no duties or responsibilities, except those duties and responsibilities expressly set forth herein. Escrow Agent may consult with counsel and shall be fully protected with respect to any action taken or omitted by it in good faith on the advise of counsel, and shall have no liability hereunder except for its gross negligence or willful misconduct. Escrow Agent may rely on any certificate, statement, request, consent, agreement or other instrument which it believes

3

in good faith to be genuine and to have been signed or presented by a proper person or persons. Escrow Agent shall not be bound by any modification to or amendment of this Agreement unless in writing and signed by the parties hereto, and if its duties as Escrow Agent hereunder are affected, unless it shall have given its prior written consent thereto. In the event that Escrow Agent shall be uncertain as to its duties or rights hereunder or shall receive instructions from the parties hereto with respect to the Deposits which, in its opinion, are in conflict with any provisions of this Agreement, Escrow Agent shall be entitled to refrain from taking any action other than to keep safely the Deposits, until it shall be directed otherwise in writing by all of the parties hereto.

(b) Notwithstanding anything herein to the contrary or which might be construed to the contrary, the Escrow Agent shall be fully and completely discharged and released from any obligation or duty with respect to the Deposits when the Escrow Agent has disbursed the same in accordance with this Agreement.

(c) If any dispute arises as to any action to be taken by the Escrow Agent or any distribution of the Deposits, or as to Escrow Agent’s rights and duties under this Agreement, or if the Escrow Agent is in doubt as to such duties, or if any attachment, garnishment, or other similar proceeding is instituted or procured as to all or any part of the Deposits, and if any writ, order, or rule of attachment, garnishment, or other similar order is levied upon or entered against any portion of the Deposits, the Escrow Agent shall have the right to: (i) interplead the Deposits, or any part thereof, with a court of competent jurisdiction and upon notifying the other parties to this Agreement of such action, all liability of the Escrow Agent under the terms of this Agreement shall terminate, except to the extent of accounting for the Deposits; or (ii) institute an action for a declaratory judgment as to the ownership of the Deposits.

(d) Escrow Agent shall not receive compensation for the performance of its duties hereunder.

(e) Escrow Agent shall be indemnified for, and held harmless against, any loss or liability (including reasonable attorneys’ fees) incurred without willful misconduct or gross negligence on the part of Escrow Agent, and arising out of or in connection with its acceptance of, or the performance of its duties and obligations under, this Agreement.

(f) Escrow Agent may resign hereunder by giving written notice thereof to the parties hereto; provided, however, that Escrow Agent shall not be permitted to resign unless and until a successor escrow agent shall have been designated and approved by all parties hereto, which approval shall not be unreasonably withheld, and shall have accepted such designation and agreed to be bound by the provisions of this Agreement. Such resignation shall then take effect upon the date specified in such notice, whereupon all duties of Escrow Agent so resigning shall cease, other than the duty to account.

4

9. Miscellaneous.

(a) Severability. If any provision in this Agreement for any reason shall be held to be illegal, invalid or unenforceable, the other provisions of this Agreement will remain in full force and effect. Any provision of this Agreement held invalid or unenforceable only in part or degree will remain in full force and effect to the extent not held invalid or unenforceable.

(b) Governing Law. This Agreement shall be governed and construed in accordance with the laws of the State of New York applicable to agreements made and to be performed entirely within such state, without regard to the conflict of laws rules thereof.

(c) Notices. Any notice or other communication required or permitted hereunder shall be in writing and shall be delivered personally by hand, telecopier or mailed (by overnight courier or registered or certified mail, postage prepaid) as follows:

5

6

Each such notice or other communication shall be effective (i) if given by telecopier, when such telecopy is transmitted to the telecopier number specified in Section 9(c) (with confirmation of transmission) or (ii) if given by any other means, when delivered at the address specified in Section 9(c). Any party by notice given in accordance with this Section 9(c) to the other party may designate another address (or telecopier number) or person for receipt of notices hereunder.

(d) Consent to Jurisdiction and Service of Process. The parties hereto irrevocably (i) agree that any suit, action or other legal proceeding arising out of this Agreement may be brought in the courts of the State of New York or the courts of the United States located in New York County, New York, (ii) consent to the jurisdiction of each court in any such suit, action or proceeding, (iii) waive any objection which they, or any of them, may have to the laying of venue of any such suit, action or proceeding in any of such courts, and (iv) waive the right to a trial by jury in any such suit, action or other legal proceeding.

(e) Counterparts. This Agreement may be executed in any number of counterparts, each of which shall be deemed to be an original as against any party whose signature appears thereon, and all of which shall together constitute one and the same instrument. This Agreement shall become binding when one or more counterparts hereof, individually or taken together, shall bear the signatures of all of the parties reflected hereon as the signatories.

[SIGNATURE PAGE FOLLOWS]

7

IN WITNESS WHEREOF, the undersigned have executed this Agreement as of the date first above written.

8

9

EXHIBIT A

Fully Executed Copy of Exchange Agreement

See Attached

10

EXHIBIT B

Fully Executed Copy of Settlement Agreement

See Attached

11

EXHIBIT C

Form of Certificate

CERTIFICATE

The undersigned, being the parties to that certain Escrow Agreement dated as of February 18, 2009 (the “Escrow Agreement”), each hereby certifies the following to WolfBlock LLP, as Escrow Agent under such Escrow Agreement, and to the other parties thereto with the understanding that that Escrow Agent will rely upon such representations in connection with its release and distribution of the Deposits pursuant to the terms and conditions of the Escrow Agreement:

1. The exchange of those certain Securities and Existing Securities, as contemplated by that certain Exchange Agreement, dated as of February 18, 2009, has occurred in accordance with the terms and conditions set forth therein.

2. All items required to be delivered in accordance with Section 7 of that certain Settlement Agreement, dated as of February 18, 2009 have either been (i) duly delivered and approved, or (ii) waived by the applicable parties thereto.

3. Subject to the satisfaction of the remaining Release Conditions set forth in Section 4 of the Escrow Agreement, the undersigned hereby authorizes Escrow Agent to release and disburse the Deposits in accordance with the provisions of Section 5 of the Escrow Agreement.

[Remainder of Page Intentionally Left Blank]

12

IN WITNESS WHEREOF, the undersigned have executed this Certificate as of                             , 2009.

13

14