AMENDMENT NO. 4 TO AMENDED AND RESTATED MASTER REPURCHASE AGREEMENT

Exhibit 10.54

EXECUTION

AMENDMENT NO. 4

TO AMENDED AND RESTATED

MASTER REPURCHASE AGREEMENT

Amendment No. 4, dated as of October 1, 2013 (this “Amendment”), among CREDIT SUISSE FIRST BOSTON MORTGAGE CAPITAL LLC (the “Buyer”), PENNYMAC CORP. (“PennyMac Corp.”) and PENNYMAC MORTGAGE INVESTMENT TRUST HOLDINGS I, LLC (“PMITH” and together with PennyMac Corp., each a “Seller” and collectively, the “Sellers”) and PENNYMAC MORTGAGE INVESTMENT TRUST (the “Guarantor”).

RECITALS

The Buyer, the Sellers and the Guarantor are parties to that certain Amended and Restated Master Repurchase Agreement, dated as of August 25, 2011 (as amended by Amendment No. 1, dated as of June 6, 2012, Amendment No. 2, dated as of March 28, 2013 and Amendment No. 3, dated as of May 8, 2013, the “Existing Repurchase Agreement”; and as further amended by this Amendment, the “Repurchase Agreement”). The Guarantor is party to that certain Guaranty (the “Guaranty”), dated as of June 8, 2011, as the same may be amended from time to time, by the Guarantor in favor of Buyer. Capitalized terms used but not otherwise defined herein shall have the meanings given to them in the Existing Repurchase Agreement and Guaranty, as applicable.

The Buyer, the Sellers and the Guarantor have agreed, subject to the terms and conditions of this Amendment, that the Existing Repurchase Agreement be amended to reflect certain agreed upon revisions to the terms of the Existing Repurchase Agreement. As a condition precedent to amending the Existing Repurchase Agreement, the Buyer has required the Guarantor to ratify and affirm the Guaranty on the date hereof.

Accordingly, the Buyer, the Sellers and the Guarantor hereby agree, in consideration of the mutual promises and mutual obligations set forth herein, that the Existing Repurchase Agreement is hereby amended as follows:

SECTION 1. PMITH Name Change.

1.1 The Buyer hereby acknowledges and Sellers hereby confirm that effective as of October 1, 2013 PMITH will have changed its legal name from “PennyMac Mortgage Investment Trust Holdings I, LLC” to “PennyMac Holdings, LLC”.

1.2 Each reference to “PennyMac Mortgage Investment Trust Holdings I, LLC” in the Existing Repurchase Agreement and each Program Agreement is hereby amended to be “PennyMac Holdings, LLC”.

-1-

SECTION 2. Conditions Precedent. This Amendment shall become effective as of the date hereof (the “Amendment Effective Date”), subject to the satisfaction of the following conditions precedent:

2.1 Delivered Documents. On the Amendment Effective Date, the Buyer shall have received the following documents, each of which shall be satisfactory to the Buyer in form and substance:

(a) this Amendment, executed and delivered by duly authorized officers of the Buyer, the Sellers and the Guarantor;

(b) a certified copy of the certificate of amendment to the certificate of formation from the jurisdiction of organization of PMITH evidencing the name change of PMITH; and

(c) such other documents as the Buyer or counsel to the Buyer may reasonably request.

2.2 UCC Financing Statements. Sellers shall permit Buyer to take all steps as may be necessary in connection with performing UCC Searches and filing duly authorized and filed Uniform Commercial Code financing statements on Form UCC-1 and UCC-3, as applicable.

SECTION 3. Representations and Warranties. Each Seller hereby represents and warrants to the Buyer that each Seller is in compliance with all the terms and provisions set forth in the Repurchase Agreement on its part to be observed or performed, and that no Event of Default has occurred and is continuing, and hereby confirms and reaffirms the representations and warranties contained in Section 13 of the Repurchase Agreement.

SECTION 4. Limited Effect. Except as expressly amended and modified by this Amendment, the Existing Repurchase Agreement shall continue to be, and shall remain, in full force and effect in accordance with its terms.

SECTION 5. Counterparts. This Amendment may be executed by each of the parties hereto on any number of separate counterparts (including by facsimile or .pdf), each of which shall be an original and all of which taken together shall constitute one and the same instrument.

SECTION 6. Severability. Each provision and agreement herein shall be treated as separate and independent from any other provision or agreement herein and shall be enforceable notwithstanding the unenforceability of any such other provision or agreement.

SECTION 7. GOVERNING LAW. THIS AMENDMENT SHALL BE GOVERNED BY, AND CONSTRUED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK WITHOUT REFERENCE TO THE CHOICE OF LAW PROVISIONS THEREOF.

SECTION 8. Reaffirmation of Guaranty. The Guarantor hereby ratifies and affirms all of the terms, covenants, conditions and obligations of the Guaranty and acknowledges and agrees that the term “Obligations” as used in the Guaranty shall apply to all of the Obligations of the Sellers to Buyer under the Repurchase Agreement and Pricing Side Letter, as amended hereby.

[Remainder of page intentionally left blank]

-2-

IN WITNESS WHEREOF, the parties have caused their names to be signed hereto by their respective officers thereunto duly authorized as of the day and year first above written.

Signature Page to Amendment No. 4 to Amended and Restated Master Repurchase Agreement