AMENDMENT NO. 1 TO AMENDED AND RESTATED LOAN AND SECURITY AGREEMENT AND CONSENT

EXHIBIT 10.65

Execution

AMENDMENT NO. 1 TO

AMENDED AND RESTATED LOAN AND SECURITY AGREEMENT AND CONSENT

AMENDMENT NO. 1 TO AMENDED AND RESTATED LOAN AND SECURITY AGREEMENT AND CONSENT, dated as of January 9, 2014 (this “Amendment No. 1”), entered into by and among Wells Fargo Bank, National Association, successor by merger to Wachovia Bank, National Association, in its capacity as agent acting for and on behalf of the parties to the Loan Agreement (as hereinafter defined) as lenders (in such capacity, “Agent”), the parties to the Loan Agreement as lenders (individually a “Lender” and collectively, “Lenders”), Perry Ellis Menswear, LLC, a Delaware limited liability company formerly known as Perry Ellis Menswear, Inc. and successor by merger to Salant Holding, LLC (“Perry Ellis Menswear”), Supreme International, LLC, a Delaware limited liability company formerly known as Supreme International, Inc. and successor by merger to Jantzen, LLC (“Supreme”; and together with Perry Ellis Menswear and any other Person that at any time after the date hereof becomes a Borrower in accordance with the terms of the Loan Agreement (as hereafter defined), each individually “Borrower” and collectively, “Borrowers”), PEI Licensing, Inc., a Delaware corporation and successor by merger to Jantzen Apparel, LLC (“PEI Licensing”), Perry Ellis International, Inc., a Florida corporation (“Parent”), Perry Ellis Real Estate, LLC, a Delaware limited liability company formerly known as Perry Ellis Real Estate Corporation (“PE Real Estate”), Perry Ellis Shared Services Corporation, a Delaware corporation (“PE Shared Services”), Supreme Realty, LLC, a Florida limited liability company and successor by merger to Supreme Real Estate I, LLC, Supreme Real Estate II LLC and Winnsboro DC, LLC (“Supreme Realty”), and Tampa DC, LLC, a Delaware limited liability company (“Tampa DC”; and together, with PEI Licensing, Parent, PE Real Estate, PE Shared Services, Supreme Realty, and any other Person that at any time after the date hereof becomes a Guarantor in accordance with the terms of the Loan Agreement, each individually a “Guarantor” and collectively, “Guarantors”).

W I T N E S S E T H :

WHEREAS, Agent, Lenders, Borrowers and Guarantors have entered into financing arrangements pursuant to which Lenders (or Agent on behalf of Lenders) have made and may make loans and advances and provide other financial accommodations to Borrowers as set forth in the Amended and Restated Loan and Security Agreement, dated as of December 2, 2011, by and among Agent, Lenders, Borrowers and Guarantors, (as the same now exists and may hereby and hereafter be amended, modified, supplemented, extended, renewed, restated, restructured, refinanced or replaced, the “Loan Agreement”, and together with all agreements, documents and instruments at any time executed and/or delivered in connection therewith or related thereto, as from time to time amended, modified, supplemented, extended, renewed, restated, or replaced, collectively, the “Financing Agreements”);

WHEREAS, Borrowers and Guarantors have requested that Agent and Lenders agree to make certain amendments to the Loan Agreement, and Agent and Lenders are willing to agree to such amendments, subject to the terms and conditions set forth in this Amendment No. 1;

WHEREAS, Borrowers and Guarantors have advised that they have consummated, effective December 30, 2013, the following mergers among the Loan Parties (each a “Loan Party Merger”, and collectively, the “Loan Party Mergers”): (a) Jantzen Apparel, LLC. with and into

PEI Licensing, with PEI Licensing as the surviving corporation, (b) Supreme Real Estate I, LLC with and into Supreme Realty, with Supreme Realty as the surviving company, (c) Supreme Real Estate II, LLC with and into Supreme Realty, with Supreme Realty as the surviving company, (d) Winnsboro DC, LLC with and into Supreme Realty, with Supreme Realty as the surviving company; (e) Salant Holding with and into Perry Ellis Menswear, with Perry Ellis Menswear as the surviving company, and (f) Jantzen, LLC with and into Supreme, with Supreme as the surviving company, and

WHEREAS, Borrowers and Guarantors have advised that they have consummated, effective December 30, 2013, the merger of Tampa DC Holdings, LLC, a Delaware limited liability company and an inactive Subsidiary of Tampa DC (“Tampa Holdings”), with and into Tampa DC, with Tampa DC as the surviving company (the “Tampa Holdings Merger”); and

WHEREAS, by this Amendment No. 1, Agent, Lenders, Borrowers and Guarantors wish to evidence such amendments and consent;

NOW, THEREFORE, in consideration of the foregoing, the mutual agreements and covenants contained herein, and other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the parties hereto agree as follows:

1. Definitions. Section 1.1 of the Loan Agreement is hereby modified as follows:

(a) Additional Definitions. Each of the following definitions shall be added to the Loan Agreement:

(i) “Amendment No. 1” shall mean Amendment No. 1 to Loan and Security Agreement, dated as of January 9, 2014, by and among Agent, Lenders, Borrowers and Guarantors.

(ii) “Amendment No. 1 Effective Date” shall mean the first date on which all of the conditions precedent to the effectiveness of Amendment No. 1 shall have been satisfied and/or waived.

(iii) “Quarterly Average Unused Line” shall have the meaning set forth in Section 3.2 herein.

(b) Amendments to Existing Definition of Applicable Margin. The definition of “Applicable Margin” is hereby amended by deleting it in its entirety and substituting the following therefor:

““Applicable Margin” means, at any time, as to the Interest Rate for Prime Rate Loans and the Interest Rate for Eurodollar Rate Loans the applicable percentage (on a per annum basis) set forth below if the sum of (a) the Quarterly Average Excess Availability for the immediately preceding fiscal quarter plus (b) the Excess Cash as of the last day of such immediately preceding fiscal quarter, is at or within the amounts indicated for such percentage:

2

(c) Interpretation. For purposes of this Amendment No. 1, unless otherwise defined herein, all capitalized terms used herein which are defined in the Loan Agreement shall have the meanings given to such terms in the Loan Agreement.

2. Letter of Credit Accommodations. Section 2.2 is hereby amended by deleting clause (e) thereof and substituting the following therefor:

“(e) Except in Agent’s discretion, with the consent of all Lenders, the amount of all outstanding Letter of Credit Accommodations and all other commitments and obligations made or incurred by Agent or any Lender in connection therewith shall not at any time exceed $30,000,000.”

3. Fees. Section 3.2 is hereby amended by deleting clause (a) thereof and substituting the following therefor:

“(a) Borrowers shall pay to Agent for the ratable benefit of Lenders monthly an unused line fee at a rate equal to the percentage (on a per annum basis) set forth below calculated upon the amount (“Quarterly Average Unused Line”) by which the Maximum Credit as then in effect exceeds the average daily principal balance of the outstanding Loans and Letter of Credit Accommodations during the immediately preceding quarter (or part thereof), while the Loan Agreement is in effect and for so long thereafter as any Obligations are outstanding. Such fee shall be payable on the first day of each month in arrears.

3

4. Term. Section 13.1 is hereby amended by deleting the first sentence of clause (a) thereof and substituting the following therefor:

“This Agreement and the other Financing Agreements shall become effective as of the Closing Date and shall continue in full force and effect for a term ending on December 1, 2018 (the “Maturity Date”) and unless sooner terminated pursuant to the terms hereof.”

5. Tampa Holdings Merger. Agent, on behalf of the Lenders, consents effective as of December 30, 2013 to the Tampa Holdings Merger.

6. Representations, Warranties and Covenants. Borrowers and Guarantors, jointly and severally, represent, warrant and covenant with and to Agent and Lenders as follows, which representations, warranties and covenants shall survive the execution and delivery hereof:

(a) this Amendment No. 1 and all other documents, agreements and instruments executed by any Borrower or Guarantor in connection herewith (together with this Amendment No. 1, the “Amendment Documents”) have been duly authorized, executed and delivered by all necessary action on the part of each Borrower and Guarantor which is a party hereto and, if necessary, their respective stockholders, and are in full force and effect as of the date hereof, and the agreements and obligations of Borrowers and Guarantors contained herein and therein constitute legal, valid and binding obligations of Borrowers and Guarantors enforceable against them in accordance with their terms except as such enforceability may be limited by (i) bankruptcy, insolvency, reorganization, moratorium or similar laws of general applicability affecting the enforcement of creditors’ rights and (ii) the application of general principles of equity (regardless of whether such enforceability is considered in a proceeding in equity or at law);

(b) neither the Amendment Documents nor any of the Receivables Exchange Program Documents nor the transactions contemplated thereby are in contravention of any applicable law, or the terms of any agreement to which any Borrower or Guarantor is a party or by which any property of any Borrower or Guarantor is bound;

(c) as of the date hereof, no Default or Event of Default exists or has occurred and is continuing; and

(d) Borrowers and Guarantors shall provide to Agent, in respect of the Loan Party Mergers and the Tampa Holding Merger, an officer’s certificate certifying as of the effective date of each Loan Party Merger and the Tampa Holding Merger compliance of such Loan Party Merger with Section 9.7(a) of the Loan Agreement, and the completion of the Tampa Holding Merger, together with any agreements, documents or instruments to be delivered in connection therewith.

7. Conditions Precedent. The terms and provisions of this Amendment No. 1 shall only be effective upon the satisfaction of each of the following conditions precedent in a manner satisfactory to Agent:

(a) Agent shall have received executed counterparts of this Amendment No. 1, duly authorized, executed and delivered by Borrowers and Guarantors and such Lenders as are required under the Loan Agreement to approve the transactions contemplated by this Amendment No. 1;

4

(b) Agent shall have received executed counterparts of the Amendment No. 1 Fee Letter, dated as of the Amendment No. 1 Effective Date, by Borrowers in favor of Agent (the “Amendment No. 1 Fee Letter”), duly authorized, executed and delivered by Borrowers;

(c) Agent shall have received from Borrowers in immediately available funds, or Agent shall have charged to any loan account of a Borrower, all of the fees set forth in the Amendment No. 1 Fee Letter; and

(d) No Default or Event of Default shall exist or have occurred and be continuing.

8. Effect of this Amendment. This Amendment No. 1 and the other Amendment Documents constitute the entire agreement of the parties with respect to the subject matter hereof and thereof, and supersede all prior oral or written communications, memoranda, proposals, negotiations, discussions, term sheets and commitments with respect to the subject matter hereof and thereof. Except as expressly provided herein, no other consents, changes or modifications to the Financing Agreements are intended or implied, and in all other respects the Financing Agreements are hereby specifically ratified, restated and confirmed by all parties hereto as of the effective date hereof. To the extent that any provision of the Loan Agreement or any of the other Financing Agreements are inconsistent with the provisions of this Amendment No. 1, the provisions of this Amendment No. 1 shall control.

9. Further Assurances. Each Borrower and Guarantor shall execute and deliver such additional documents and take such additional action as may be reasonably requested by Agent to effectuate the provisions and purposes of this Amendment No. 1.

10. Release of Claims. No Borrower or Guarantor has any actual or potential claim or cause of action against Agent or any Lender with respect to any matters relating to the Financing Agreements and related transactions through the date hereof, and hereby waives and releases any right to assert same.

11. Governing Law. The validity, interpretation and enforcement of this Amendment No. 1 and any dispute arising out of the relationship between the parties hereto, whether in contract, tort, equity or otherwise, shall be governed by the internal laws of the State of Florida but excluding any principles of conflicts of law or other rule of law that would cause the application of the law of any jurisdiction other than the laws of the State of Florida.

12. Binding Effect. This Amendment No. 1 shall be binding upon and inure to the benefit of each of the parties hereto and their respective successors and assigns.

13. Counterparts. This Amendment No. 1 may be executed in any number of counterparts, each of which shall be an original, but all of which taken together shall constitute one and the same agreement. Delivery of an executed counterpart of this Amendment No. 1 by telefacsimile or other electronic means shall have the same force and effect as the delivery of an

5

original executed counterpart of this Amendment No. 1. Any party delivering an executed counterpart of this Amendment No. 1 by telefacsimile or other electronic means shall also deliver an original executed counterpart, but the failure to do so shall not affect the validity, enforceability or binding effect of such agreement

[Signature Pages Follow]

6

IN WITNESS WHEREOF, the parties hereto have caused this Amendment No. 1 to be duly executed and delivered by their authorized officers as of the day and year first above written.

[SIGNATURES CONTINUED ON NEXT PAGE]

[SIGNATURES CONTINUED FROM PREVIOUS PAGE]

[SIGNATURES CONTINUED ON NEXT PAGE]

[SIGNATURES CONTINUED FROM PREVIOUS PAGE]

[SIGNATURES CONTINUED ON NEXT PAGE]

[SIGNATURES CONTINUED FROM PREVIOUS PAGE]

[SIGNATURES CONTINUED ON NEXT PAGE]

[SIGNATURES CONTINUED FROM PREVIOUS PAGE]

[SIGNATURES CONTINUED ON NEXT PAGE]

[SIGNATURES CONTINUED FROM PREVIOUS PAGE]

Schedule I

Commitments