EMPLOYMENT AGREEMENT

Exhibit 10.5

EMPLOYMENT AGREEMENT

This Employment Agreement (“Agreement”) is made as of June, 9, 2008 between CASUAL MALE RETAIL GROUP, INC., a Delaware corporation with an office at 555 Turnpike Street, Canton, Massachusetts, 02021 (the “Company”, which term includes its subsidiaries and affiliates), and Francis Chane (the “Executive”) having an address at 45 Saddleworthy Way, Middleboro, MA 02346.

WITNESSETH:

WHEREAS, the Company desires that Executive serve as Vice President Distribution and Logistics, and Executive desires to be so employed by the Company.

WHEREAS, Executive and the Company desire to set forth in writing the terms and conditions of the Executive’s employment with the Company from the date hereof.

NOW, THEREFORE, in consideration of the promises and the mutual promises, representations and covenants herein contained, the parties hereto agree as follows:

The Company hereby employs Executive and Executive hereby accepts such employment, subject to the terms and conditions herein set forth. Executive shall hold the office of Vice President Distribution and Logistics.

The term of employment under this Agreement shall begin on the date set forth above (the “Effective Date”) and shall continue until terminated by either party as hereinafter set forth (such period of employment being referred to herein as the “Term”), subject to prior termination in accordance with the terms hereof.

(a) As compensation for the employment services to be rendered by Executive hereunder, the Company agrees to pay to Executive, and Executive agrees to accept, payable in equal bi-weekly installments in accordance with Company practice, an annual base salary of One Hundred Eighty Thousand Dollars and No Cents ($180,000.00).

(b) In addition to the annual base salary, Executive is eligible to participate in the Company’s Annual Incentive Plan. Such incentive shall be determined and payable in accordance with the Company’s incentive program in effect at the time, subject to change from year to year in the Company’s sole discretion. Executive will participate in the Company’s incentive program at a rate of 35% (50% max) of Executive’s actual annual base earnings based upon the Executive’s individual performance component and/or the Company’s targeted EBITDA. The actual award under the incentive program, if any, may be more or less than the target and will be based on Executive’s performance and the performance of the Company and payment will be made in accordance with the terms of the incentive program then in effect.

(c) In addition, Executive is eligible to participate in the Company’s Long Term Incentive Plan (“LTIP”). Such incentive shall be determined and distributable in accordance with the terms and conditions as described in the LTIP documents in effect at the time of the award, subject to change from year to year in the Company’s sole discretion. Executive will participate in the Company’s LTIP at an incentive rate of 70%, of Executive’s combined actual annual base earnings, for the incentive period, based upon the Company’s targeted performance as defined in the LTIP documents in effect at the time of the award.

The Company shall pay or reimburse Executive, in accordance with the Company’s policies and procedures and upon presentment of suitable vouchers, for all reasonable business and travel expenses, which may be incurred or paid by Executive in connection with his employment hereunder. Executive shall comply with such restrictions and shall keep such records as the Company may reasonably deem necessary to meet the requirements of the Internal Revenue Code of 1986, as amended from time to time, and regulations promulgated thereunder.

(a) Executive shall be entitled to such vacations and to participate in and receive any other benefits customarily provided by the Company to its management (including any profit sharing, pension, 401(k), short and long-term disability insurance, medical and dental insurance and group life insurance plans in accordance with the terms of such plans), all as determined from time to time by the Compensation Committee of the Board of Directors in its discretion.

(b) The Company will, during the term of Executive’s employment hereunder, provide Executive with an automobile allowance in the total amount of Seven Thousand Two Hundred Dollars and No Cents ($7200.00) annually, in equal bi-weekly payments in accordance with the Company’s normal payroll practices. Executive shall pay and be responsible for all insurance, repairs and maintenance costs associated with operating the automobile. Executive is responsible for his gasoline, unless the gasoline expense is reimbursable under the Company’s policies and procedures.

(c) Executive will be eligible to participate in the Company’s annual performance appraisal process.

(a) Executive shall perform such duties and functions consistent with his position as Vice President Distribution and Logistics and/or as the Company shall from time to time determine and Executive shall comply in the performance of his duties with the policies of, and be subject to the direction of the Company.

2

(b) During the Term of this Agreement, Executive shall devote substantially all of his time and attention, vacation time and absences for sickness excepted, to the business of the Company, as necessary to fulfill his duties. Executive shall perform the duties assigned to him with fidelity and to the best of his ability. Notwithstanding anything herein to the contrary, and subject to the foregoing, Executive shall not be prevented from accepting positions in outside charitable organizations so long as such activities do not interfere with Executive’s performance of his duties hereunder and do not violate Section 9 hereof.

(c) The principal location at which the Executive shall perform his duties hereunder shall be at the Company’s offices in Canton, Massachusetts or at such other location as may be temporarily designated from time to time by the Company. Notwithstanding the foregoing, Executive shall perform such services at such other locations as may be required for the proper performance of his duties hereunder, and Executive recognizes that such duties may involve travel.

(a) Executive’s employment hereunder may be terminated by the Company at any time:

(i) upon the determination that Executive’s performance of his duties has not been fully satisfactory for any reason which would not constitute justifiable cause (as hereinafter defined) or for other business reasons necessitating termination which do not constitute justifiable cause, in either case upon thirty (30) days’ prior written notice to Executive; or

(ii) upon the determination that there is justifiable cause (as hereinafter defined) for such termination.

(b) Executive’s employment shall terminate upon:

(i) the death of Executive;

(ii) the “total disability” of Executive (as hereinafter defined in Subsection (c) herein) pursuant to Subsection (h) hereof; or

(iii) Executive’s resignation of employment.

(c) For the purposes of this Agreement, the term “total disability” shall mean Executive is physically or mentally incapacitated so as to render Executive incapable of performing the essentials of Executive’s job, even with reasonable accommodation, as reasonably determined by the Company, which determination shall be final and binding.

(d) For the purposes hereof, the term “justifiable cause” shall mean: any failure or refusal to perform any of the duties pursuant to this Agreement or any breach of this Agreement by the Executive; Executive’s breach of any material written policies, rules or regulations which have been adopted by the Company; Executive’s repeated failure to perform his duties in a satisfactory manner; Executive’s performance of any act or his failure to act, as to which if

3

Executive were prosecuted and convicted, a crime or offense involving money or property of the Company or its subsidiaries or affiliates, or a crime or offense constituting a felony in the jurisdiction involved, would have occurred; any unauthorized disclosure by Executive to any person, firm or corporation of any confidential information or trade secret of the Company or any of its subsidiaries or affiliates; any attempt by Executive to secure any personal profit in connection with the business of the Company or any of its subsidiaries and affiliates; or the engaging by Executive in any business other than the business of the Company and its subsidiaries and affiliates which interferes with the performance of his duties hereunder. Upon termination of Executive’s employment for justifiable cause, this Agreement shall terminate immediately and Executive shall not be entitled to any amounts or benefits hereunder other than such portion of Executive’s annual base salary and reimbursement of expenses pursuant to Section 5 hereof as have been accrued through the date of his termination of employment.

(e) If the Company terminates this Agreement without “justifiable cause” as provided in Subsection 7 (a)(i), the Company shall pay Executive his then current base salary for five months after the effectiveness of such termination, payable in equal payments in accordance with the Company’s customary payroll practices. However, if Executive is employed or retained, as an employee, independent contractor, consultant or in any other capacity or if he is offered another position by the Company at a comparable salary (“New Employment”) prior to or during the time he receives payment under this Subsection or Subsection 3 (b), the Company is entitled to a credit for all sums paid or earned by Executive during this period of time or which he could have earned had he accepted the comparable position by the Company. The Executive must make a good faith effort to find New Employment and mitigate the amount of money to be paid by the Company to Executive under this Subsection or Subsection 3(b). Executive also agrees to immediately notify the Company if and when he is offered another position and/or accepts another position. The Company will pay any amount due and owing in accordance with the payment schedule in 3(a), until paid in full. Any payment pursuant to this paragraph 7(e) is contingent upon Executive’s execution of a general release and separation agreement in a form acceptable to the Company and will be in lieu of payments to which Executive might have been entitled under any other severance plan of the Company.

(f) If Executive shall die during the term of his employment hereunder, this Agreement shall terminate immediately. In such event, the estate of Executive shall thereupon be entitled to receive such portion of Executive’s base annual salary and reimbursement of expenses pursuant to Section 4 as have been accrued through the date of his death.

(g) Upon Executive’s “total disability”, the Company shall have the right to terminate Executive’s employment. Any termination pursuant to this Subsection (g) shall be effective on the earlier of (i) the date 30 days after which Executive shall have received written notice of the Company’s election to terminate or (ii) the date he begins to receive long-term disability insurance benefits under the policy provided by the Company pursuant to Section 5 hereof.

(h) Upon the resignation of Executive in any capacity, that resignation will be deemed to be a resignation from all offices and positions that Executive holds with respect to the Company and any of its subsidiaries and affiliates. In the event of Executive’s resignation, he shall be entitled to receive only such portion of his annual base salary and reimbursement of expenses pursuant to Section 4 as have been accrued through the date of his resignation.

4

(i) Change of Control. In the event the Executive’s employment with the Company is terminated by the Company during the Term without justifiable cause within twelve months following a Change of Control of the Company occurring during the Term then, in such event, the Company shall pay Executive an amount equal to twelve (12) months of base annual salary in effect at the time of the termination, payable in equal payments in accordance with the Company’s customary payroll practices, which amount will be subject to mitigation in accordance with Section 7(e) above. For the purposes of the foregoing, Change of Control shall have the meaning set forth in the Company’s 2006 Incentive Stock Option Plan (without regard to any subsequent amendments thereto). Any payment pursuant to this paragraph 7(i) is contingent upon Executive’s execution of a general release and separation agreement in a form acceptable to the Company and will be in lieu of payments to which Executive might have been entitled under paragraph 7(e) of this Agreement.

(a) Executive represents and warrants that he is free to enter into this Agreement and to perform the duties required hereunder, and that there are no employment contracts or understandings, restrictive covenants or other restrictions, whether written or oral, preventing the performance of his duties hereunder.

(b) Executive agrees to submit to a medical examination and to cooperate and supply such other information and documents as may be required by any insurance company in connection with the Company’s obtaining life insurance on the life of Executive, and any other type of insurance or fringe benefit as the Company shall determine from time to time to obtain.

(c) Executive represents and warrants that he has never been convicted of a felony and he has not been convicted or incarcerated for a misdemeanor within the past five years, other than a first conviction for drunkenness, simple assault, speeding, minor traffic violations, affray, or disturbance of the peace.

(d) Executive represents and warrants that he has never been a party to any judicial or administrative proceeding that resulted in a judgement, decree, or final order (i) enjoining him from future violations of, or prohibiting any violations of any federal or state securities law, or (ii) finding any violations of any federal or state securities law.

(e) Executive represents and warrants that he has never been accused of any impropriety in connection with any employment;

Any breach of any of the above representations and warranties is “justifiable cause” for termination under Section 7(d) of this Agreement.

5

Executive agrees that during his employment by the Company and during the two (2) year period following the termination of Executive’s employment hereunder (the “Non-compete and Non-solicitation Period””), Executive shall not:

(a) directly or indirectly, as owner, partner, joint venturer, stockholder, employee, broker, agent, principal, trustee, corporate officer, director, licensor, or in any capacity whatsoever, engage in, become financially interested in, be employed by, render any consultation or business advice with respect to, accept any competitive business on behalf of, or have any connection with any business which is competitive with products or services of the Company or any subsidiaries and affiliates, in any geographic area in which the Company or any of its subsidiaries or affiliates are then conducting or proposing to conduct business, including, without limitation, the United States of America and its possessions, Canada and Europe; provided, however, that Executive may own any securities of any corporation which is engaged in such business and is publicly owned and traded but in an amount not to exceed at any one time one percent (1%) of any class of stock or securities of such corporation;

(b) directly or indirectly, request or cause any suppliers or customers with whom the Company or any of its subsidiaries and affiliates has a business relationship to cancel or terminate any such business relationship with the Company or any of its subsidiaries and affiliates or otherwise compromise the Company’s good will or solicit, hire, offer employment to, or in any manner encourage current employees of the Company to leave its employ. Employee further agrees that the same prohibition applies to any former employees who were employed by the Company at any time during Employee’s final six months of employment with the Company.

(c) If any portion of the restrictions set forth in this Section 9 should, for any reason whatsoever, be declared invalid by a court of competent jurisdiction, the validity or enforceability of the remainder of such restrictions shall not thereby be adversely affected. For the purposes of this Section 9, a business competitive with the products and services of the Company (or such subsidiaries and affiliates) is limited to a specialty retailer which primarily distributes, sells or markets so-called “big and tall” apparel of any kind for men or which utilizes the “big and tall” retail or wholesale marketing concept as part of its business.

(d) Executive acknowledges that the Company conducts business throughout the world, that Executive’s duties and responsibilities on behalf of the Company are of a worldwide nature, that its sales and marketing prospects are for continued expansion throughout the world and therefore, the territorial and time limitations set forth in this Section 9 are reasonable and properly required for the adequate protection of the business of the Company and its subsidiaries and affiliates. In the event any such territorial or time limitation is deemed to be unreasonable by a court of competent jurisdiction, Executive agrees to the reduction of the territorial or time limitation to the area or period which such court shall deem reasonable.

(e) The existence of any non-material claim or cause of action (a “non-material” claim or cause of action is defined as a claim or cause of action which results from something other than a material breach of the terms and provisions of this Agreement by the Company) by

6

Executive against the Company or any subsidiary or affiliate shall not constitute a defense to the enforcement by the Company or any subsidiary or affiliate of the foregoing restrictive covenants, but such claim or cause of action shall be litigated separately.

(f) In the event that Employee breaches any of the provisions of Paragraph 9 of this Agreement, the Non-Compete and Non-Solicitation period shall be tolled until such breach has been duly cured.

(g) In the event that Employee believes that employment otherwise in violation of this Agreement would not harm the Company’s legitimate business interests, Employee may request Company to waive the restrictions contained in this Agreement. Any such request shall be made in writing to the Senior Vice President, Human Resources at the Company and shall identify the business with which Employee seeks to associate and describe the duties that Employee seeks to perform. The Company has the sole discretion whether to grant such a waiver and no waiver of any restrictions under this Agreement shall be effective unless in writing and signed by the CFO/COO or CEO of the Company.

(a) Upon execution of this Agreement and thereafter, Executive shall promptly and fully disclose to the Company, and with all necessary detail for a complete understanding of the same, all existing and future developments, know-how, discoveries, inventions, improvements, concepts, ideas, writings, formulae, processes and Methods (whether copyrightable, patentable or otherwise) made, received, conceived, acquired or written during working hours, or otherwise, by Executive (whether or not at the request or upon the suggestion of the Company) during the period of his employment with, or rendering of advisory or consulting services to, the Company or any of its subsidiaries and affiliates, solely or jointly with others, in or relating to any activities of the Company or its subsidiaries and affiliates known to him as a consequence of his employment or the rendering of advisory and consulting services hereunder (collectively the “Subject Matter”).

(b) Executive hereby assigns and transfers, and agrees to assign and transfer, to the Company, all his rights, title and interest in and to the Subject Matter, and Executive further agrees to deliver to the Company any and all drawings, notes, specifications and data relating to the Subject Matter, and to execute, acknowledge and deliver all such further papers, including applications for copyrights or patents, as may be necessary to obtain copyrights and patents for any thereof in any and all countries and to vest title thereto to the Company. Executive shall assist the Company in obtaining such copyrights or patents during the term of this Agreement, and at any time thereafter on reasonable notice and at mutually convenient times, and Executive agrees to testify in any prosecution or litigation involving any of the Subject Matter; provided, however, that Executive shall be compensated in a timely manner at the rate of $250 per day (or portion thereof), plus out-of-pocket expenses incurred in rendering such assistance or giving or preparing to give such testimony if it is required after the termination of this Agreement.

7

(a) Executive acknowledges that the Company possesses certain confidential and propriety information that has been or may be revealed to him or learned by Executive during the course of Executive’s employment with the Company and that it would be unfair to use that information or knowledge to compete with or to otherwise disadvantage the Company. Executive shall not, during the term of this Agreement or at any time following termination of this Agreement, directly or indirectly, disclose or permit to be known (other than as is required in the regular course of his duties (including without limitation disclosures to the Company’s advisors and consultants), as required by law (in which case Executive shall give the Company prior written notice of such required disclosure) or with the prior written consent of the Board of Directors of the Company, to any person, firm, corporation, or other entity, any confidential information acquired by him during the course of, or as an incident to, his employment or the rendering of his advisory or consulting services hereunder, relating to the Company or any of its subsidiaries and affiliates, the directors of the Company or its subsidiaries and affiliates, any supplier or customer of the Company or any of their subsidiaries and affiliates, or any corporation, partnership or other entity owned or controlled, directly or indirectly, by any of the foregoing, or in which any of the foregoing has a beneficial interest, including, but not limited to, the business affairs of each of the foregoing. Such confidential information shall include, but shall not be limited to, proprietary technology, trade secrets, patented processes, research and development data, know-how, market studies and forecasts, financial data, competitive analyses, pricing policies, employee lists, personnel policies, the substance of agreements with customers, suppliers and others, marketing or dealership arrangements, servicing and training programs and arrangements, supplier lists, customer lists and any other documents embodying such confidential information. This confidentiality obligation shall not apply to any confidential information, which is or becomes publicly available other than pursuant to a breach of this Section 11(a) by Executive.

(b) All information and documents relating to the Company and its affiliates as herein above described (or other business affairs) shall be the exclusive property of the Company, and Executive shall use commercially reasonable best efforts to prevent any publication or disclosure thereof. Upon termination of Executive’s employment with the Company, all documents, records, reports, writings and other similar documents containing confidential information, including copies thereof then in Executive’s possession or control shall be returned and left with the Company.

Executive agrees that if he breaches, or threatens to commit a breach of, any of the provisions of Sections 9, 10 or 11 (the “Restrictive Covenants”), the Company shall have, in addition to, and not in lieu of, any other rights and remedies available to the Company under law and in equity, the right to have the Restrictive Covenants specifically enforced by a court of competent jurisdiction, it being agreed that any breach or threatened breach of the Restrictive Covenants would cause irreparable injury to the Company and that money damages would not provide an adequate remedy to the Company. Notwithstanding the foregoing, nothing herein shall constitute a waiver by Executive of his right to contest whether a breach or threatened

8

breach of any Restrictive Covenant has occurred. The Company shall be entitled to recover from Executive all attorneys’ fees and expenses reasonably incurred in establishing a breach of this agreement. The damages, attorneys’ fees and costs shall be in addition to and not in lieu of any preliminary injunctive relief that may be available to the Company.

No amendment or alteration of the terms of this Agreement shall be valid unless made in writing and signed by both of the parties hereto.

This Agreement shall be governed by, and construed and enforced in accordance with the substantive laws of The Commonwealth of Massachusetts, without regard to its principles of conflicts of laws.

The holding of any provision of this Agreement to be invalid or unenforceable by a court of competent jurisdiction shall not affect any other provision of this Agreement, which shall remain in full force and effect.

Any notices required or permitted to be given hereunder shall be sufficient if in writing, and if delivered by hand or courier, or by a reputable overnight courier with receipt acknowledgement capabilities, or sent by certified mail, return receipt requested, to the addresses set forth above or such other address as either party may from time to time designate in writing to the other, and shall be deemed given as of the date of the delivery or at the expiration of three days in the event of a mailing.

It is agreed that a waiver by either party or a breach of any provision of this Agreement shall not operate, or be construed as a waiver of any subsequent breach by that same party.

This Agreement contains the entire agreement of the parties with respect to the subject matter hereof and shall be binding upon and inure to the benefit of the parties hereto and their respective legal representatives, heirs, distributors, successors and assigns and supersedes any and all prior agreements between the parties whether oral or written. This Agreement may not be modified except upon further written agreement executed by both parties. Executive agrees that the Company may in its sole discretion, during the term of Executive’s employment with the Company and thereafter, provide copies of this Agreement (or excerpts of the Agreement) to others, including businesses or entities that may employ, do business with, or

9

consider employing Executive in the future. Executive further agrees that any subsequent change or changes in his duties, compensation or areas of responsibility shall in no way affect the validity of this Agreement or otherwise render inapplicable any of the provisions of paragraphs 9 through 12 of this Agreement, which shall remain in full force and effect except as may be modified by a subsequent written agreement.

Except as otherwise expressly provided herein, the termination of Executive’s employment hereunder or the expiration of this Agreement shall not affect the enforceability of Sections 7, 8, 9, 10, 11 and 12 hereof.

Any and all disputes arising under or in connection with this Agreement shall be resolved in accordance with this Section 20.

The parties shall attempt to resolve any dispute, controversy or difference that may arise between them through good faith negotiations. In the event the parties fail to reach resolution of any such dispute within thirty (30) days after entering into negotiations, either party may proceed to institute action in any state or federal court located within the Commonwealth of Massachusetts and each party consents to the personal jurisdiction of any such state or federal court.

The parties agree to execute and deliver all such further documents, agreements and instruments and take such other and further action as may be necessary or appropriate to carry out the purposes and intent of this Agreement.

The Section headings appearing in this Agreement are for the purposes of easy reference and shall not be considered a part of this Agreement or in any way modify, amend or affect its provisions.

10

This Agreement may be executed in one or more counterparts, each of which shall be deemed an original and all of which together shall constitute one and the same agreement.

IN WITNESS WHEREOF, the parties hereto have executed this Agreement, under seal, as of the date and year first above written.

11