RESTRICTIVE COVENANTS AGREEMENT

Exhibit 10.4

EXECUTION COPY

RESTRICTIVE COVENANTS AGREEMENT

THIS RESTRICTIVE COVENANTS AGREEMENT (“Agreement”) is entered into as of June 1, 2010 (the “Effective Date”) by and among The First American Corporation, a California corporation (“FAC”) and the First American Financial Corporation, a Delaware corporation (“FinCo”).

Recitals

A. FAC and certain of its Subsidiaries and Affiliates, including FinCo, are entering into a transaction whereby FAC will spin off FinCo into a separate, independent, publicly traded company (the “Spin-off”) pursuant to a Separation and Distribution Agreement between FAC and FinCo dated as of June 1, 2010 (the “Separation Agreement”; capitalized terms used herein and not otherwise defined herein shall have the meanings given them in the Separation Agreement). FAC will retain its information solutions business and continue to operate as a publicly traded company following the Spin-off.

B. FinCo is receiving and, as a result of the Spin-off, FinCo’s Subsidiaries will receive significant benefits from the Spin-off.

C. After the Spin-off, FAC will conduct the Business (as defined below) either on its own or through its direct or indirect Subsidiaries or Affiliates, including First American Real Estate Tax Service LLC, a Delaware limited liability company (“FARETS”).

D. It is a condition to the consummation of the Spin-off that FinCo enters into this Agreement and the restrictive covenants contained herein.

NOW, THEREFORE, in consideration of the promises and the terms and conditions set forth in this Agreement, the parties agree as follows:

(a) Noncompete. FinCo covenants and agrees that it (1) shall not, (2) shall ensure that each other member of the FinCo Group shall not, and (3) shall use commercially reasonable best efforts to ensure that each of its Affiliates shall not, for ten (10) years after the Effective Date (the “Prohibition Period”), without the prior written consent of FAC, directly or indirectly:

(i) conduct or engage in the Business in the United States of America; or

(ii) manage, operate, control or participate in the management, operation or control of any Person (1) which is conducting or engaged in the

Business in the United States of America and (2) which derives 25% or more of its consolidated revenues from the Business measured as of the end of the fiscal year of such Person immediately preceding the first date on which FinCo manages, operates, controls or participates in the management, operation or control of such Person;

provided, however, that FinCo shall not be deemed to be engaged in the activities described above in this Section 1(a) or otherwise in breach of this Section 1(a) as a result of (1) the acquisition and continued operation of a Person conducting or engaging in the Business (a “Target Competitor”) so long as the Target Competitor derived 15% or less of its total consolidated revenues (measured as of the end of such Person’s most recently completed fiscal year) from the Business or (2) the acquisition and continued operation of FinCo by a Person conducting or engaging in the Business (a “Competitive Acquiror”) so long as the Competitive Acquiror derived 15% or less of its total consolidated revenues (measured as of the end of such Person’s most recently completed fiscal year and on a pro forma basis to include the revenues of FinCo) from the Business; provided, further, that if FinCo acquires a Target Competitor that derives more than 15% of its total consolidated revenues (measured as of the end of such Person’s most recently completed fiscal year) from the Business or if FinCo is acquired by a Competitive Acquiror that derives more than 15% of its total consolidated revenues (measured as of the end of such Person’s most recently completed fiscal year and on a pro forma basis to include the revenues of FinCo) from the Business, then FinCo shall not be deemed to breach this Section 1(a) if it (or the Competitive Acquiror, if applicable) engages in commercially reasonable best efforts to divest itself of businesses engaged in the Business. For purposes of this Section 1(a), an “acquisition” shall include an acquisition by stock purchase, merger, consolidation or the purchase of all or substantially all of the assets of a Person.

(b) Certain Defined Terms:

(i) As used herein, “Affiliate” means a Person existing on or after the Effective Date (and, for the avoidance of doubt, subsequent to the Spin-off), that directly or indirectly, through one or more intermediaries, controls, is controlled by or is under common control with, a referenced Person.

(ii) As used herein, the “Business” means the provision to financial institutions engaged in the United States of America in the business of making, originating, acquiring or servicing loans secured by Relevant Real Property (“Mortgage Lenders”) of: (a) services, monitoring and reporting on the status of tax obligations with respect to Relevant Real Property, including, without limitation, delinquencies in the payment of tax obligations with respect to such real property, and handling related customer service inquiries, (b) data procured from taxing authorities relating to tax obligations with respect to Relevant Real Property, which data are used by FARETS for the purpose of monitoring the status of tax obligations with respect to Relevant Real Property on the Mortgage Lenders’ behalf; and (c) services providing for the collection and reimbursement of funds from Mortgage Lenders for property tax impound and nonimpound accounts and the electronic or manual processing and disbursement of such funds to taxing authorities, including check processing (other than in connection with the provision of escrow or closing services in connection with the origination of a loan);

2

provided, however, that the “Business” shall not include any business engaged in by the FinCo Group as of the Effective Date, including, without limitation, any business engaged in by DataTrace Information Services LLC, a Delaware corporation, or its subsidiaries as of the Effective Date (which includes, for the avoidance of doubt, the provision to any Person of tax certificates); provided, further, that the “Business” shall not include any business not engaged in by FARETS or First American Commercial Real Estate Services, Inc. as of the Effective Date.

(iii) As used herein, “Change in Control” means the happening of any of the following:

(A) The consummation of a merger or consolidation of FAC with or into another Person or any other corporate reorganization, if fifty percent (50%) or more of the combined voting power of the continuing or surviving Person’s securities outstanding immediately after such merger, consolidation, or other reorganization is owned by persons who were not shareholders of FAC immediately prior to such merger, consolidation, or other reorganization;

(B) The sale, transfer, or other disposition of all or substantially all of FAC’s assets or the complete liquidation or dissolution of FAC;

(C) A change in the composition of the board of directors of FAC (the “Board”) occurring within a two (2) year period, as a result of which fewer than a majority of the directors are Incumbent Directors. “Incumbent Directors” shall mean directors who either: (i) are directors of FAC as of the moment immediately following the Spin-off or (ii) are elected, or nominated for election, to the Board with the affirmative votes of at least a majority of the Incumbent Directors at the time of such election or nomination (but, in either case, shall not include an individual not otherwise an Incumbent Director whose election or nomination is in connection with an actual or threatened proxy contest relating to the election of directors to FAC);

(D) Any transaction as a result of which any person or group is or becomes the “beneficial owner” (as defined in Rule 13d-3 under the Securities Exchange Act of 1934), directly or indirectly, of securities of FAC representing at least twenty-five percent (25%) of the total voting power of FAC’s then outstanding voting securities. For purposes of this paragraph, the term “person” shall have the same meaning as when used in Sections 13(d) and 14(d) of the Securities Exchange Act of 1934, but shall exclude: (i) a trustee or other fiduciary holding securities under an employee benefit plan of FAC or of a subsidiary of FAC; (ii) so long as a person does not thereafter increase such person’s beneficial ownership of the total voting power represented by FAC’s then outstanding voting securities, a person whose beneficial ownership of the total voting power represented by FAC’s then outstanding voting securities increases to twenty-five percent (25%) or more as a result of the acquisition of voting securities of FAC by FAC which reduces the number of such voting securities then outstanding; or (iii) so long as a person does not thereafter increase such person’s beneficial

3

ownership of the total voting power represented by FAC’s then outstanding voting securities, a person that acquires directly from FAC securities of FAC representing at least twenty-five percent (25%) of the total voting power represented by FAC’s then outstanding voting securities;

(E) The sale, transfer or other disposition of FARETS to a party or group of parties that are not Affiliates of FAC, pursuant to which such party or parties (i) acquire, directly or indirectly, a majority of the interests of FARETS, or (ii) acquire, directly or indirectly, all or substantially all of the assets of FARETS; or

(F) The sale, transfer or other disposition, directly or indirectly, of the Person conducting, or a substantial portion of the assets used in conducting, the Business.

For the avoidance of doubt, a transaction shall not constitute a Change in Control if its sole purpose is to change the state of FAC’s incorporation or to create a holding company that will be owned in substantially the same proportions by the persons who held FAC’s securities immediately before such transaction.

(iv) As used herein, “Relevant Real Property” means, with reference to a Mortgage Lender, real property located in the United States of America that is the subject of a loan that is made, originated, acquired or serviced by such Mortgage Lender.

(c) Exceptions to Noncompete. Notwithstanding Section 1(a) hereof, FinCo and/or any of its Affiliates may own (1) equity securities of any Person engaged in the Business so long as the aggregate holdings of FinCo and its Affiliates in such Person, whether held directly or indirectly, does not exceed ten percent (10%) of the total outstanding equity securities (on a fully converted basis), (2) equity securities of any Person which derives 15% or less of its consolidated revenues from the Business (measured as of the end of the fiscal year of such Person immediately preceding the date on which FinCo first acquires ten percent (10%) or more of the total outstanding equity securities of such Person and, with respect to any subsequent acquisitions of equity securities, as of the end of the fiscal year of such Person immediately preceding the date on which FinCo acquires such equity securities), and (3) any securities, of any type and in any amount, of FAC.

2. Remedies, Acknowledgement and Construction.

(a) Equitable Relief. In the event of any actual or threatened breach of the provisions of this Agreement, the parties acknowledge and agree that FAC, in addition to any other remedies available to it in law or in equity for such beach or threatened breach, shall be entitled to seek an award of equitable relief, including injunctive relief and specific performance, without the necessity of posting a bond or proving irreparable harm or actual damages.

(b) Acknowledgment of Reasonableness. FinCo has carefully considered the provisions of this Agreement and agrees that the restrictions set forth herein are fair and reasonable and required for protection of the legitimate interests of FAC and are a material and necessary part of the transactions contemplated by the Spin-off. FinCo further agrees that the

4

restrictions are reasonable in scope, area and time, and will not prevent FinCo from pursuing other non-competitive business ventures or otherwise cause a financial hardship to FinCo.

3. Consideration. FinCo agrees that it is receiving good and valuable consideration for entering into this Agreement, which consideration includes, among other things, the receipt of consideration pursuant to the terms of the Spin-off. FinCo acknowledges that FAC has relied upon the covenants contained in this Agreement and that such covenants are conditions to, and a material part of, the willingness of FAC to consummate the transactions contemplated by the Spin-off.

4. Binding Effect; Benefits; Assignment. This Agreement shall inure to the benefit of, and shall be binding upon, the parties hereto and their respective successors and permitted assigns. This Agreement shall not be assignable by any party hereto without the prior written consent of the other parties hereto.

5. Termination Upon a Change in Control. Notwithstanding any other provision to the contrary, this Agreement and all rights and obligations hereunder shall terminate upon the occurrence of a Change in Control.

6. Notices. Any notices required or permitted under, or otherwise in connection with this Agreement shall be in writing and shall be deemed to have been duly given when delivered in person, upon confirmation of receipt when transmitted by facsimile transmission, by email, on receipt after dispatch by registered or certified mail, postage prepaid, addressed, or on the next business day if transmitted by national overnight courier, in each case as follows:

If to FAC, addressed to:

The First American Corporation

4 First American Way

Santa Ana, California 92707

Attn: Chief Executive Officer

With a copy to the General Counsel at the same address;

if to FinCo, addressed to:

First American Financial Corporation

1 First American Way

Santa Ana, California 92707

Attn: Chief Executive Officer,

with a copy to the General Counsel at the same address.

Any party may change the address to which notices, requests, demands, claims and other communications hereunder are to be delivered by giving the other parties notice in the manner herein set forth.

5

7. Entire Agreement. This Agreement constitutes the entire agreement of the parties and supersede all prior agreements and undertakings, both written and oral, between the parties, or any of them, with respect to the subject matter hereof.

8. Amendments and Waivers. This Agreement may not be modified or amended except by an instrument or instruments in writing signed by the party against whom enforcement of any such modification or amendment is sought. Each party hereto may, by an instrument in writing, waive compliance by the other party with any term or provision of this Agreement on the part of such other party hereto to be performed or complied with. The waiver by any party hereto of a breach of any term or provision of this Agreement shall not be construed as a waiver of any subsequent breach.

9. Section and Other Headings. The section and other headings contained in this Agreement are for reference purposes only and shall not be deemed to be a part of this Agreement or to control or affect the meaning or construction of any provision of this Agreement.

10. Governing Law; Consent to Jurisdiction.

(a) This Agreement and the transactions contemplated hereby, and all disputes between the parties under or related to the Agreement or the facts and circumstances leading to its execution, whether in contract, tort or otherwise, shall be governed by and construed in accordance with the laws of the State of New York, applicable to contracts executed in and to be performed entirely within the state by residents of such state.

(b) Each of the parties hereto hereby irrevocably and unconditionally submits, for itself and its property, to the exclusive jurisdiction of any court of the State of New York and the federal court of the United States of America, in each case sitting in the County of New York, and any appellate court from any thereof, in any action or proceeding arising out of or relating to this Agreement or the agreements delivered in connection herewith or the transactions contemplated hereby or thereby or for recognition or enforcement of any judgment relating thereto, and each of the parties hereby irrevocably and unconditionally (i) agrees not to commence any such action or proceeding except in such courts, (ii) agrees that any claim in respect of any such action or proceeding may be heard and determined in such New York state court or, to the extent permitted by law, in such federal court, (iii) waives, to the fullest extent it may legally and effectively do so, any objection which it may now or hereafter have to the laying of venue of any such action or proceeding in any such New York state or federal court, and (iv) waives, to the fullest extent permitted by law, the defense of an inconvenient forum to the maintenance of such action or proceeding in any such New York state or federal court. Each of the parties hereto agrees that a final judgment in any such action or proceeding shall be conclusive and may be enforced in other jurisdictions by suit on the judgment or in any other manner provided by law. Each party to this Agreement irrevocably consents to service of process in the manner provided for notices in Section 5. Nothing in this Agreement shall affect the right of any party to this Agreement to serve process in any other manner permitted by law.

11. WAIVER OF JURY TRIAL. EACH OF THE PARTIES HERETO HEREBY IRREVOCABLY WAIVES TO THE FULLEST EXTENT PERMITTED BY LAW ALL

6

RIGHT TO TRIAL BY JURY IN ANY ACTION, PROCEEDING OR COUNTERCLAIM (WHETHER BASED ON CONTRACT, TORT OR OTHERWISE) ARISING OUT OF OR RELATING TO THIS AGREEMENT OR THE ACTIONS OF EACH PARTY IN THE NEGOTIATION, ADMINISTRATION, PERFORMANCE AND ENFORCEMENT HEREOF.

12. Termination. This Agreement shall automatically terminate without any action on the part of the parties hereto or any person if the Spin-off is terminated for any reason in accordance with its terms. FinCo shall not have any liability to FAC under this Agreement with respect to any of the covenants and agreements set forth herein if the Spin-off is not consummated.

13. Counterparts. This Agreement may be executed in one or more counterparts, and by the different parties hereto in separate counterparts, each of which when executed shall be deemed to be an original but all of which taken together shall constitute one and the same agreement. A signature on a copy of this Agreement received by either party by facsimile transmission or electronically transmitted portable document format (“PDF”) is binding upon the executing party and shall be deemed an original signature. The parties shall treat a photocopy or electronic image of such facsimile or PDF document as a duplicate original.

Remainder of page intentionally left blank. Signature page follows.

7

IN WITNESS WHEREOF, FAC and FinCo have caused this Agreement to be executed as of the Effective Date by their respective officers thereunto duly authorized.