ANWORTH MORTGAGE ASSET CORPORATION CHANGE IN CONTROL AND ARBITRATION AGREEMENT

Exhibit 10.4

ANWORTH MORTGAGE ASSET CORPORATION

CHANGE IN CONTROL AND ARBITRATION AGREEMENT

THIS CHANGE IN CONTROL AND ARBITRATION AGREEMENT is made and entered into as of June 27, 2006, by and between Anworth Mortgage Asset Corporation, a Maryland corporation (the “Company”) and Thad M. Brown (“Employee”).

WHEREAS, the Board of Directors (the “Board”) of the Company has determined that, in the event of a possible, threatened or pending sale or other change in control of the Company, it is imperative that the Company and the Board be able to rely upon Employee to continue in Employee’s position, and that the Company be able to receive and rely upon Employee’s advice, if requested, as to the best interests of the Company and its shareholders without concern that Employee might be distracted by the personal uncertainties and risks created by any such possible transactions; and

WHEREAS, in connection with such a change in control, Employee may, in addition to Employee’s regular duties, be called upon to assist in the assessment of any such possible transactions, advise management and the Board as to whether such proposals would be in the best interests of the Company and its shareholders, and to take such other actions as the Board might determine to be appropriate; and

WHEREAS, the Board and the Compensation Committee have authorized the Company to enter into an agreement with Employee providing benefits as set forth herein;

NOW, THEREFORE, to assure the Company that it will have the continued dedication of Employee and the availability of Employee’s advice and counsel through the occurrence of any Change in Control of the Company, and to induce Employee to remain in the employ of the Company, and for other good and valuable consideration, the Company and Employee agree as follows:

1. Change in Control Benefits. In the event that a Change in Control (as defined on Annex A and made a part hereof) occurs, then the Company shall pay to Employee a lump sum amount (the “Additional Amount”) equal to (i) twelve (12) months annual base salary in effect on the date of the Change in Control, plus (ii) the average annual incentive compensation received by the Employee for the two complete fiscal years (or such lesser number of years as the employee has been employed by the Company) of the Company immediately prior to the effective date of the Change in Control, plus (iii) the average annual bonus received by the Employee for the two complete fiscal years (or such lesser number of years as the employee has been employed by the Company) of the Company immediately prior to the effective date of the Change in Control. The Additional Amount shall be paid upon the date that the Change in Control occurs.

2. Welfare Benefits.

(a) If Employee’s employment with the Company terminates during the twelve (12) month period immediately following a Change in Control, Company shall provide to Employee (and his spouse and other qualified dependents) all Welfare Benefits that Company

provided to Employee (and his spouse and qualified dependents) immediately prior to the Change in Control for a period of twelve (12) months following Employee’s termination of employment with Company. Should the continuation of any Welfare Benefits to be provided to Employee hereunder following the termination of the Employee’s employment be unavailable under the Company’s benefit plans for any reason, the Company shall reimburse Employee his cost of obtaining such benefits under substantially similar plans from similar third-party providers.

(b) If prior to the Change in Control Employee was required to contribute towards the cost of a Welfare Benefit as a condition of receiving such Welfare Benefit, the Employee may be required to continue contributing towards the cost of such Welfare Benefit under the same terms and conditions as applied to the Employee immediately prior to the Change in Control in order to receive such Welfare Benefit.

3. Equity Awards.

(a) The Company has granted Employee equity awards, consisting of options to purchase Company common stock that are currently outstanding, but not yet exercisable in whole or in part, and shares of restricted Company common stock that have been issued and are outstanding, but which are subject to forfeiture rights (collectively with any other equity awards of any type that may be granted to Employee in the future, “Equity Awards”). The currently outstanding stock options and any future stock options the Company grants to Employee are referred to hereinafter as “Stock Options” and the currently outstanding restricted stock and any future restricted stock the Company grants to Employee are hereinafter referred to as the “Restricted Stock.”

(b) Notwithstanding the provisions of any agreement(s) pursuant to which any Equity Awards are or have been granted, in the event that a Change in Control occurs, all Equity Awards shall become fully vested. In the case of Stock Options, all of such Stock Options held by Employee shall become fully vested and immediately exercisable. In the case of Restricted Stock, all of the Restricted Stock held by Employee shall become fully vested and any forfeiture rights to which such Restricted Stock is held shall terminate in full.

4. Other Employee Benefits. The benefits provided to Employee hereunder shall not be affected by or reduced because of any other benefits (including, but not limited to, salary, bonus, pension, stock option or restricted stock plan) to which Employee may be entitled by reason of his employment with the Company or the termination of his employment with the Company, and no other such benefit by reason of such employment shall be so affected or reduced because of the benefits bestowed by this Agreement.

5. Withholding. All amounts payable by the Company hereunder shall be subject to all federal, state, local and other withholdings and employment taxes as required by applicable law.

6. No Mitigation; No Offset. The parties hereto agree that Employee shall not be required to mitigate damages in respect of any benefit or payment due under this Agreement or in respect of any damage award as a result of the Company’s breach of this Agreement, nor shall any such benefit or award be offset by any future compensation or income received by Employee

2

from any other source. The Company shall not have the right to offset against its obligations hereunder or against any such damage award any amounts payable by Employee to Company for any reason.

7. Arbitration of Claims. The following arbitration provisions shall apply to any claim brought by Employee or the Company after the date of this Agreement even if the facts upon which the claim is based arose prior to the execution of this Agreement:

(a) Claims Covered by this Agreement. To the maximum extent permitted by law, the Company and Employee mutually consent to the resolution by arbitration of all claims or causes of action that the Company may have against Employee or that Employee may have against the Company or against its officers, directors, employees, or agents in the capacity as such or otherwise (collectively “claims”). The claims covered by this Agreement include, but are not limited to, claims for breach of any contract or covenant (express or implied); tort claims; claims for discrimination (including, but not limited to, race, sex, sexual harassment, or any type of unlawful harassment, religion, national origin, age, marital status, medical condition, disability or sexual orientation); claims for wrongful termination in violation of public policy; and claims for violation of any federal, state, or other governmental law, statute, regulation or ordinance, including, but not limited to, all claims arising under Title VII of the Civil Rights Act of 1969, as amended, the Age Discrimination in Employment Act of 1967, the Americans with Disabilities Act, the California Fair Employment & Housing Act, the California Labor Code, the Consolidated Omnibus Budget Reconciliation Act of 1985, the Fair Labor Standards Act or Employee Retirement Income Security Act.

(b) Claims Not Covered by the Agreement. Claims Employee may have for workers’ compensation, unemployment compensation benefits or wage and hour claims within the jurisdiction of the California Labor Commissioner are not covered by this Agreement. Notwithstanding the fact that Employee is not required to arbitrate such claims, he may, if he so chooses, submit wage and hour claims to binding arbitration pursuant to this Agreement. Also not covered are claims by either party for injunctive and/or other equitable relief, as to which the parties understand and agree that either party may seek and obtain relief from a court of competent jurisdiction.

(c) Required Notice of All Claims. The Company and Employee agree that the aggrieved party must give written notice of any claim to the other party. Written notice to the Company, or its officers, employees or agents, shall be sent to the Company’s Chief Executive Officer. The written notice shall identify and describe the nature of all claims asserted and detail the facts upon which such claims are based. The notice shall be sent to the other party by certified or registered mail, return receipt requested.

(d) Arbitration Procedures. The Company and Employee agree that, except as provided in this Agreement, any arbitration shall be in accordance with and under the auspices and rules of the American Arbitration Association (hereinafter the “Arbitration Service”). The arbitration shall take place in Los Angeles County, California, unless the parties mutually agree to conduct the arbitration in a different location. The arbitrator shall be selected by the mutual agreement of the parties. If the parties cannot agree on a neutral arbitrator, Employee first, and then the Company, will alternately strike names from a list provided by the Arbitration Service until only one name remains. The arbitrator shall have exclusive authority to resolve any dispute

3

relating to the interpretation, applicability, enforceability or formation of this Agreement, including but not limited to any claim that all or any part of this Agreement is void or voidable. The arbitrator shall apply the applicable statute of limitations to any claim, taking into account compliance with Section 8(c) of this Agreement. The arbitrator shall issue a written opinion and award, which shall be signed and dated. The arbitrator shall be permitted to award those remedies that are available under applicable law. The arbitrator’s decision regarding the claims shall be final and binding upon the parties. The arbitrator’s award shall be enforceable in any court having jurisdiction thereof.

(e) Arbitration Fees and Costs; Attorneys’ Fees. Employee will be required to pay an arbitration fee to initiate the arbitration equal to what he would be charged as a first appearance fee in court. The Company shall advance the remaining fees and costs of the arbitrator. However, to the extent permissible under the law, and following the arbitrator’s ruling on the matter, the arbitrator may rule that the arbitrator’s fees and costs be distributed in an alternative manner. The arbitrator’s award in any arbitration brought pursuant to the provisions of this Agreement shall provide for the prevailing party to recover from the other party the prevailing party’s reasonable attorneys’ fees relating to such action.

(f) Requirements for Modification or Revocation. This agreement to arbitrate shall survive the termination of Employee’s employment with the Company. It can only be revoked or modified by a writing signed by the parties that specifically states an intent to revoke or modify this Agreement.

8. Entire Agreement; Effect of Prior Agreements. This is the complete agreement of the parties on the subjects set forth herein, including Change in Control pay and arbitration of disputes. This Agreement supersedes any prior or contemporaneous oral or written understanding on such subjects. No party is relying on any representations, oral or written, on the subject of the effect, enforceability, or meaning of this Agreement, except as specifically set forth in this Agreement. In the event of a conflict between any of the terms of this Agreement and any of the terms of any agreement pursuant to which Employee was granted an Equity Award, the terms of this Agreement shall prevail.

9. Amendment. This Agreement may not be amended without the prior written consent of both Employee and the Company.

10. No Right to Continued Employment. This Agreement does not constitute a contract of employment, does not change the status of the Employee’s employment and does not change the Company’s policies regarding termination of employment. Nothing in this Agreement shall be deemed to give Employee the right to be retained in the service of the Company or to deny the Company any right it may have to discharge or demote him at any time; provided, however, that any termination of employment of Employee, or any removal of Employee as an employee or officer of the Company primarily in contemplation of a Change in Control shall not be effective to deny Employee the benefits of this Agreement, including without limitation Sections 2, 3 and 4 hereof. No provision of this Agreement shall in any way limit, restrict or prohibit Employee’s right to terminate employment with the Company.

11. Severability. If a court or other body of competent jurisdiction determines that any provision of this Agreement is invalid or unenforceable, that provision will be adjusted

4

rather than voided, if possible, so that it is enforceable to the maximum extent possible, or, if it is not possible to so adjust such provision, this Agreement shall be construed in all respects as if such invalid or unenforceable provision were omitted. The invalidity and unenforceability of any particular provision of this Agreement shall not affect any other provision hereof, and all other provisions of the Agreement shall be valid and enforceable to the fullest extent possible.

12. Successors.

(a) The Company will require any successor, whether direct or indirect, by purchase, merger, consolidation or otherwise, to all or substantially all of the business and/or assets of the Company to expressly assume and agree to perform this Agreement in the same manner and to the same extent that the Company would be required to perform it if no such succession had taken place.

(b) This Agreement shall inure to the benefit of, and be enforceable by, Employee’s personal or legal representatives, executors, administrators, successors, heirs, distributees, devisees and legatees.

13. Governing Law. This Agreement shall be governed by and construed in accordance with the laws of the State of California without regard or reference to the rules of conflicts of law that would require the application of the laws of any other jurisdiction.

14. Notices. All notices and other communications under this Agreement shall be in writing and mailed, telecopied, or delivered by hand or by a nationally recognized courier service guaranteeing overnight delivery to a party, at the following address (or to such other address as such party may have specified by notice given to the other party pursuant to this provision):

If to Employee, to:

Thad M. Brown

958 Blue Mountain Circle

Westlake Village, CA 91362

Telephone: (818)  ###-###-####

Facsimile: (310)  ###-###-####

5

If to the Company, to:

Anworth Mortgage Asset Corporation

1299 Ocean Avenue, Suite 250

Santa Monica, CA 90401

Facsimile:  ###-###-####

Attention: Chairman

with a copy to:

Manatt, Phelps & Phillips, LLP

11355 W. Olympic Blvd.

Los Angeles, CA 90064

Telephone: (310)  ###-###-####

Facsimile: (310)  ###-###-####

Attention: Mark J. Kelson

All such notices and communications shall, when mailed, telecopied, or delivered, be effective three days after deposit in the United States mail, telecopied with confirmation of receipt, or delivered by hand to the addressee or one day after delivery to the courier service.

15. Counterparts. This Agreement may be executed in more than one counterpart, each of which shall be deemed to be an original, and all such counterparts together shall constitute one and the same instrument.

16. Consultation With Counsel. Employee acknowledges that he has had a full and complete opportunity to consult with counsel and other advisors of his own choosing concerning the terms, enforceability and implications of this Agreement, and that the Company has not made any representations or warranties to Employee concerning the terms, enforceability or implications of this Agreement other than as reflected in this Agreement.

6

IN WITNESS WHEREOF, the parties hereto have duly executed this Agreement, effective as of the date set forth in the first paragraph hereof.

7

ANNEX A

Change in Control

A “Change in Control” as used in the Agreement of which this Annex is a part shall mean the first to occur of any of the following:

(a) Any “Person” (as such term is used in Sections 13(d) and 14(d) of the Securities Exchange Act of 1934 (the “Exchange Act”) (other than the Company or any trustee or other fiduciary holding securities under an Employee benefit plan of the Company; or any company owned, directly or indirectly, by the stockholders of the Company in substantially the same proportions as their ownership of the stock of the Company), or more than one Person acting as a group, becomes the “beneficial owner” (as defined in Rule 13d–3 under the Exchange Act), directly or indirectly, of securities of the Company (not including in the securities beneficially owned by such Person any securities acquired directly from the Company or from a transferor in a transaction expressly approved or consented to by the Board of Directors) representing more than 50% of the combined voting power of the Company’s then outstanding securities; the acquisition of additional securities by any Person or Persons acting as a group who immediately prior to such acquisition already is the beneficial owner of more than 50% of the combined voting power of the Company’s then outstanding securities is not a Change in Control; or

(b) During any period of not more than 12 consecutive months (not including any period prior to the execution of this Agreement), individuals who at the beginning of such period constitute the Board of Directors and any new director (other than a director designated by a person who has entered into an agreement with the Company to effect a transaction described in clause (a), (c) or (d) of this section), (i) whose election by the Board of Directors or nomination for election by the Company’s stockholders was approved by a vote of at least a majority of the directors then still in office who either were directors at the beginning of the period or whose election or nomination for election was previously so approved or (ii) whose election is to replace a person who ceases to be a director due to death, disability or age, cease for any reason to constitute a majority thereof; or

(c) The effective date of any merger or consolidation of the Company with any other corporation or entity (after all requisite shareholder, applicable regulatory and other approvals and consents have been obtained), other than (i) a merger or consolidation which would result in the voting securities of the Company outstanding immediately prior thereto continuing to represent (either by remaining outstanding or by being converted into voting securities of the surviving entity), in combination with the ownership of any trustee or other fiduciary holding securities under an Employee benefit plan of the Company, at least 65% of the combined voting power of the voting securities of the Company or such surviving entity outstanding immediately after such merger or consolidation, or (ii) a merger or consolidation effected to implement a recapitalization of the Company (or similar transaction) in which no Person acquires more than 35% of the combined voting power of the Company’s then outstanding securities; or (iii) a merger or consolidation of the Company with one or more Persons that are related to the Company immediately prior to the consolidation or merger. For purposes of this provision, Persons are “related” if one of them owns, directly or indirectly, at

least 50% of the total value or voting power of the other or a third Person owns, directly or indirectly, at least 50% of the total value or voting power of each of them; or

(d) The sale or disposition by the Company of all or substantially all of the Company’s assets, to one or more persons that are not related (as defined in clause (c) above) to the Company immediately prior to the sale or transfer.

This definition of Change in Control is intended to comply with Section 409A of the Internal Revenue Code of 1986, as amended, and Treasury Regulations thereunder and interpretations thereof, and shall be interpreted accordingly in construing this Agreement.

2