AMENDMENT NO. 1 TO MASTER REPURCHASE AGREEMENT

Exhibit 10.80

EXECUTION

AMENDMENT NO. 1

TO MASTER REPURCHASE AGREEMENT

Amendment No. 1, dated as of May 8, 2013 (this “Amendment”), among CREDIT SUISSE FIRST BOSTON MORTGAGE CAPITAL LLC (the “Buyer”), PENNYMAC OPERATING PARTNERSHIP, L.P. (the “Seller”) and PENNYMAC MORTGAGE INVESTMENT TRUST (the “Guarantor”).

RECITALS

The Buyer, the Seller and the Guarantor are parties to that certain Master Repurchase Agreement, dated as of September 28, 2012 (as amended, the “Existing Repurchase Agreement”; and as further amended by this Amendment, the “Repurchase Agreement”). The Guarantor is a party to that certain Guaranty (the “Guaranty”), dated as of September 28, 2012, as the same may be further amended from time to time, by the Guarantor in favor of Buyer. Capitalized terms used but not otherwise defined herein shall have the meanings given to them in the Existing Repurchase Agreement and Guaranty, as applicable.

The Buyer, the Seller and the Guarantor have agreed, subject to the terms and conditions of this Amendment, that the Existing Repurchase Agreement be amended to reflect certain agreed upon revisions to the terms of the Existing Repurchase Agreement. As a condition precedent to amending the Existing Repurchase Agreement, the Buyer has required the Guarantor to ratify and affirm the Guaranty on the date hereof.

Accordingly, the Buyer, the Seller and the Guarantor hereby agree, in consideration of the mutual promises and mutual obligations set forth herein, that the Existing Repurchase Agreement is hereby amended as follows:

SECTION 1. Definitions. Section 2 of the Existing Repurchase Agreement is hereby amended by:

1.1 deleting the definitions of “Aged 75 Day Loan”, “Aged 90 Day Loan”, “Aging Limit”, “Mortgage Loan” and “Termination Date” in their entirety and replacing them with the following:

“Aged 75 Day Loan” means a Mortgage Loan other than a Jumbo Mortgage Loan which has been subject to a Transaction hereunder for a period of greater than 60 days but not greater than 75 days.

“Aged 90 Day Loan” means a Mortgage Loan other than a Jumbo Mortgage Loan which has been subject to a Transaction hereunder for a period of greater than 75 days but not greater than 90 days.

“Aging Limit” means (i) with respect to Aged Loans other than Aged Jumbo Loans, 90 days and (ii) with respect to Aged Jumbo Loans, 60 days.

- 1 -

“Mortgage Loan” means any closed Conforming Mortgage Loan, FHA Loan, VA Loan, Jumbo Mortgage Loan or Conforming High LTV Loan, which is a fixed or floating-rate, one-to-four-family residential mortgage loan evidenced by a promissory note and secured by a first lien mortgage, which satisfies the requirements set forth in the Underwriting Guidelines and Section 13.b hereof; provided, however, that, except with respect to Conforming High LTV Loans and as expressly approved in writing by Buyer, Mortgage Loans shall not include any “high-LTV” loans (i.e., a mortgage loan having a loan-to-value ratio in excess of (a) with respect to FHA Loans or VA Loans, 97%, (b) with respect to Conforming Mortgage Loans, 95% (provided that Conforming Mortgage Loans, other than Conforming High LTV Loans, with an LTV of 80% or higher must be covered by primary mortgage insurance) or (c) such lower percentage set forth in the Underwriting Guidelines).

“Termination Date” means the earliest of (a) the Rolling Termination Date; provided that if the Buyer does not deliver a Rolling Termination Notice on or before November 1, 2013, the Termination Date shall be October 31, 2014 and (b) the date of the occurrence of an Event of Default.

1.2 adding the definitions of “Aged Jumbo Loan”, “Jumbo Mortgage Loan”, “Restricted Cash”, “Rolling Termination Date” and “Rolling Termination Notice” in their proper alphabetical order:

“Aged Jumbo Loan” means a Jumbo Mortgage Loan which is an Aged 60 Day Loan.

“Jumbo Mortgage Loan” means a Mortgage Loan with an original principal balance in an amount in excess of the then applicable conventional conforming limits, including general limits and high-cost area limits, for Mortgaged Properties securing Mortgage Loans in such county or local area; provided, however, that Jumbo Mortgage Loans shall not include any Mortgage Loan with an original principal balance in excess of $2,000,000.

“Restricted Cash” means for any Person, any amount of cash of such Person that is contractually required to be set aside, segregated or otherwise reserved.

“Rolling Termination Date” means, with respect to any date, the date which is 364 days from such date; provided, that on and after the date, if any, on which the Buyer delivers to the Seller a Rolling Termination Notice (the “Rolling Termination Notice Date”) the Rolling Termination Date shall be fixed at the date that is 364 days following the Rolling Termination Notice Date.

“Rolling Termination Notice” means written notice that the Buyer shall no longer roll the Rolling Termination Date forward.

SECTION 2. Covenants. Section 14 of the Existing Repurchase Agreement is hereby amended by deleting Section 14.dd in its entirety and replacing it with the following:

“dd. Financial Covenants. Seller and Guarantor shall at all times comply with all financial covenants and/or financial ratios set forth below.

- 2 -

SECTION 3. Conditions Precedent. This Amendment shall become effective as of the date hereof (the “Amendment Effective Date”), subject to the satisfaction of the following conditions precedent:

3.1 Delivered Documents. On the Amendment Effective Date, the Buyer shall have received the following documents, each of which shall be satisfactory to the Buyer in form and substance:

(a) this Amendment, executed and delivered by duly authorized officers of the Buyer, the Seller and the Guarantor;

(b) Amendment No. 2, dated as of the date hereof to that certain Pricing Side Letter dated September 28, 2012, among Buyer, Seller and Guarantor; and

(c) such other documents as the Buyer or counsel to the Buyer may reasonably request.

SECTION 4. Representations and Warranties. Seller hereby represents and warrants to the Buyer that it is in compliance with all the terms and provisions set forth in the Repurchase Agreement on its part to be observed or performed, and that no Event of Default has occurred and is continuing, and hereby confirms and reaffirms the representations and warranties contained in Section 13 of the Repurchase Agreement.

SECTION 5. Limited Effect. Except as expressly amended and modified by this Amendment, the Existing Repurchase Agreement shall continue to be, and shall remain, in full force and effect in accordance with its terms.

SECTION 6. Counterparts. This Amendment may be executed by each of the parties hereto on any number of separate counterparts (including by facsimile or .pdf), each of which shall be an original and all of which taken together shall constitute one and the same instrument.

- 3 -

SECTION 7. Severability. Any provision of this Amendment which is prohibited or unenforceable in any jurisdiction shall, as to such jurisdiction, be ineffective to the extent of such prohibition or unenforceability without invalidating the remaining provisions hereof, and any such prohibition or unenforceability in any jurisdiction shall not invalidate or render unenforceable such provision in any other jurisdiction.

SECTION 8. GOVERNING LAW. THIS AMENDMENT SHALL BE GOVERNED BY, AND CONSTRUED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK WITHOUT REFERENCE TO THE CHOICE OF LAW PROVISIONS THEREOF.

SECTION 9. Reaffirmation of Guaranty. The Guarantor hereby ratifies and affirms all of the terms, covenants, conditions and obligations of the Guaranty and acknowledge and agree that the term “Obligations” as used in the Guaranty shall apply to all of the Obligations of Seller to Buyer under the Pricing Side Letter and the Repurchase Agreement, as amended hereby.

[Remainder of page intentionally left blank]

- 4 -

IN WITNESS WHEREOF, the parties have caused their names to be signed hereto by their respective officers thereunto duly authorized as of the day and year first above written.

Signature Page to Amendment No. 1 to Master Repurchase Agreement