AMENDMENT NO. 4 TO MASTER REPURCHASE AGREEMENT

Exhibit 10.97

AMENDMENT NO. 4

TO MASTER REPURCHASE AGREEMENT

Amendment No. 4 to Master Repurchase Agreement, dated as of October 31, 2014 (this “Amendment”), among Credit Suisse First Boston Mortgage Capital LLC (the “Buyer”), PennyMac Operating Partnership, L.P. (the “Seller”) and PennyMac Mortgage Investment Trust (the “Guarantor”).

RECITALS

The Buyer, the Seller and the Guarantor are parties to that certain Master Repurchase Agreement, dated as of September 28, 2012 (as amended by Amendment No. 1, dated as of May 8, 2013, Amendment No. 2, dated as of December 31, 2013, and Amendment No. 3, dated as of January 10, 2014, the “Existing Repurchase Agreement”; and as further amended by this Amendment, the “Repurchase Agreement”) and the related Pricing Side Letter, dated as of September 28, 2012 (as amended, restated, supplemented or otherwise modified from time to time, the “Pricing Side Letter”). The Guarantor is a party to that certain Guaranty (as amended, restated, supplemented or otherwise modified from time to time, the “Guaranty”), dated as of September 28, 2012, by the Guarantor in favor of Buyer. Capitalized terms used but not otherwise defined herein shall have the meanings given to them in the Existing Repurchase Agreement and Guaranty, as applicable.

The Buyer, the Seller and the Guarantor have agreed, subject to the terms and conditions of this Amendment, that the Existing Repurchase Agreement be amended to reflect certain agreed upon revisions to the terms of the Existing Repurchase Agreement. As a condition precedent to amending the Existing Repurchase Agreement, the Buyer has required the Guarantor to ratify and affirm the Guaranty on the date hereof.

Accordingly, the Buyer, the Seller and the Guarantor hereby agree, in consideration of the mutual promises and mutual obligations set forth herein, that the Existing Repurchase Agreement is hereby amended as follows:

SECTION 1. Applicability. Section 1 of the Existing Repurchase Agreement is hereby amended by deleting such section in its entirety and replacing it with the following:

From time to time the parties hereto may enter into transactions in which Seller agrees to transfer to Buyer Mortgage Loans (as hereinafter defined) on a servicing released basis against the transfer of funds by Buyer, with a simultaneous agreement by Buyer to transfer to Seller such Mortgage Loans on a servicing released basis at a date certain or on demand, against the transfer of funds by Seller. This Agreement is a commitment by Buyer to engage in the Transactions as set forth herein up to the Maximum Committed Purchase Price; provided, that Buyer shall have no commitment to enter into any Transaction requested that would result in the aggregate Purchase Price of then-outstanding Transactions exceeding the Maximum Committed Purchase Price, and in no event shall the aggregate Purchase Price of outstanding Transactions exceed the Maximum Purchase Price at any time. Each such transaction shall be referred to herein

-1-

as a “Transaction” and, unless otherwise agreed in writing, shall be governed by this Agreement, including any supplemental terms or conditions contained in any annexes identified herein, as applicable hereunder.

SECTION 2. Definitions. Section 2 of the Existing Repurchase Agreement is hereby amended by:

2.1 deleting the definition of “Termination Date” in its entirety and replacing it with the following:

“Termination Date” means the earliest of (a) October 30, 2015, and (b) the date of the occurrence of an Event of Default.

2.2 adding the following definition in its proper alphabetical order:

“Maximum Committed Purchase Price” has the meaning assigned to such term in the Pricing Side Letter.

SECTION 3. Program; Initiation of Transactions. Section 3 of the Existing Repurchase Agreement is hereby amended by deleting subsection (a) in its entirety and replacing it with the following:

a. From time to time, Buyer will purchase from Seller certain Mortgage Loans that have been originated or acquired by Seller from an Underlying Repurchase Counterparty pursuant to an Underlying Repurchase Transaction. This Agreement is a commitment by Buyer to enter into Transactions with Seller up to an aggregate amount equal to the Maximum Committed Purchase Price. This Agreement is not a commitment by Buyer to enter into Transactions with Seller for amounts exceeding the Maximum Committed Purchase Price, but rather, sets forth the procedures to be used in connection with periodic requests for Buyer to enter into Transactions with Seller. Seller hereby acknowledges that, beyond the Maximum Committed Purchase Price, Buyer is under no obligation to agree to enter into, or to enter into, any Transaction pursuant to this Agreement. All Purchased Mortgage Loans shall exceed or meet the Underwriting Guidelines, and shall be serviced by a Servicer. The aggregate Purchase Price of Purchased Mortgage Loans subject to outstanding Transactions shall not exceed the Maximum Purchase Price.

SECTION 4. Representations and Warranties. Section 13 of the Existing Repurchase Agreement is hereby amended by deleting subsection (11) in its entirety and replacing it with the following:

(11) Litigation. Except as set forth on Exhibit E, there is no action, proceeding or investigation pending with respect to which either Seller or Guarantor has received service of process or, to the best of Seller’s or Guarantor’s knowledge threatened against it before any court, administrative agency or other tribunal (A) asserting the invalidity of any Program Agreement, (B) seeking to prevent the consummation of any of the transactions contemplated any Program Agreement, (C) making a claim individually or in the aggregate in an amount greater than $10,000,000, (D) which requires filing with the

-2-

Securities and Exchange Commission in accordance with the 1934 Act or any rules thereunder or (E) which might materially and adversely affect the validity of the Mortgage Loans or the performance by it of its obligations under, or the validity or enforceability of any Program Agreement.

SECTION 5. Covenants. Section 14 of the Existing Repurchase Agreement is hereby amended by deleting subsection (a) in its entirety and replacing it with the following:

a. Litigation. Seller and Guarantor, as applicable, will promptly, and in any event within ten (10) days after service of process on any of the following, give to Buyer notice of all litigation, actions, suits, arbitrations, investigations (including, without limitation, any of the foregoing which are threatened or pending) or other legal or arbitrable proceedings affecting Seller, Guarantor or any of their Subsidiaries or affecting any of the Property of any of them before any Governmental Authority that (i) questions or challenges the validity or enforceability of any of the Program Agreements or any action to be taken in connection with the transactions contemplated hereby, (ii) makes a claim individually or in the aggregate in an amount greater than $10,000,000, or (iii) which, individually or in the aggregate, if adversely determined, could be reasonably likely to have a Material Adverse Effect with respect to Seller or Guarantor. Seller and Guarantor, as applicable, will promptly provide notice of any judgment, which with the passage of time, could cause an Event of Default hereunder.

SECTION 6. Events of Default. Section 15 of the Existing Repurchase Agreement is hereby amended by deleting subsection (k) in its entirety and replacing it with the following:

k. Judgment. A final judgment or judgments for the payment of money in excess of $10,000,000 in the aggregate shall be rendered against the Seller or any of its Affiliates by one or more courts, administrative tribunals or other bodies having jurisdiction and the same shall not be satisfied, discharged (or provision shall not be made for such discharge) or bonded, or a stay of execution thereof shall not be procured, within 30 days from the date of entry thereof.

SECTION 7. Represenations and Warranties. Part 2 of Schedule 1 of the Existing Repurchase Agreement is hereby amended by deleting paragraph (ff) in its entirety and replacing it with the following:

(ff) Litigation. There is no action, proceeding or investigation pending with respect to which the Underlying Repurchase Counterparty has received service of process or, to the best of such Underlying Repurchase Counterparty’s knowledge threatened against it before any court, administrative agency or other tribunal (A) asserting the invalidity of any Underlying Repurchase Document, (B) seeking to prevent the consummation of any of the transactions contemplated any Underlying Repurchase Document, (C) making a claim individually or in the aggregate in an amount greater than $10,000,000, (D) which requires filing with the Securities and Exchange Commission in accordance with the 1934 Act or any rules thereunder or (E) which might materially and adversely affect the validity of the Mortgage Loans or the performance by it of its

-3-

obligations under, or the validity or enforceability of any Underlying Repurchase Document.

SECTION 8. Conditions Precedent. This Amendment shall become effective as of the date hereof (the “Amendment Effective Date”), subject to the satisfaction of the following conditions precedent:

8.1 Delivered Documents. On the Amendment Effective Date, the Buyer shall have received the following documents, each of which shall be satisfactory to the Buyer in form and substance:

(a) this Amendment, executed and delivered by duly authorized officers of the Buyer, the Seller and the Guarantor;

(b) Amendment No. 7 to that certain Pricing Side Letter, dated as of the date hereof, executed and delivered by duly authorized officers of the Buyer, the Seller and the Guarantor; and

(c) such other documents as the Buyer or counsel to the Buyer may reasonably request.

SECTION 9. Representations and Warranties. Seller hereby represents and warrants to the Buyer that it is in compliance with all the terms and provisions set forth in the Repurchase Agreement on its part to be observed or performed, and that no Event of Default has occurred and is continuing, and hereby confirms and reaffirms the representations and warranties contained in Section 13 of the Repurchase Agreement.

SECTION 10. Limited Effect. Except as expressly amended and modified by this Amendment, the Existing Repurchase Agreement shall continue to be, and shall remain, in full force and effect in accordance with its terms.

SECTION 11. Counterparts. This Amendment may be executed by each of the parties hereto on any number of separate counterparts (including by facsimile or .pdf), each of which shall be an original and all of which taken together shall constitute one and the same instrument.

SECTION 12. Severability. Any provision of this Amendment which is prohibited or unenforceable in any jurisdiction shall, as to such jurisdiction, be ineffective to the extent of such prohibition or unenforceability without invalidating the remaining provisions hereof, and any such prohibition or unenforceability in any jurisdiction shall not invalidate or render unenforceable such provision in any other jurisdiction.

SECTION 13. GOVERNING LAW. THIS AMENDMENT SHALL BE GOVERNED BY, AND CONSTRUED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK WITHOUT REFERENCE TO THE CHOICE OF LAW PROVISIONS THEREOF.

SECTION 14. Reaffirmation of Guaranty. The Guarantor hereby ratifies and affirms all of the terms, covenants, conditions and obligations of the Guaranty and acknowledge and agree

-4-

that the term “Obligations” as used in the Guaranty shall apply to all of the Obligations of Seller to Buyer under the Pricing Side Letter and the Repurchase Agreement, as amended hereby.

[Remainder of page intentionally left blank]

-5-

IN WITNESS WHEREOF, the parties have caused their names to be signed hereto by their respective officers thereunto duly authorized as of the day and year first above written.

Signature Page to Amendment No. 4 to Master Repurchase Agreement