W I T N E S S E T H:

Page 19

Exhibit 10.1               

     THIS AGREEMENT (the “Agreement”), is made and entered into as of the 29th day of August, 2003 (the “Effective Date”) by and between The Cato Corporation, a Delaware corporation (the “Company”), and Wayland H. Cato, Jr. (“Wayland Cato”).

W I T N E S S E T H:

     WHEREAS, Wayland Cato is an employee, director and non-executive Chairman of the Board of the Company; and

     WHEREAS, the Company and Wayland Cato have negotiated and agreed on the terms of this Agreement providing for his resignation as an employee and director of the Company and for the ongoing obligation of the parties following the effective date of this Agreement and his retirement from the Company.

     NOW, THEREFORE, the parties, intending legally to be bound, hereby agree to the mutual terms and conditions set forth below:

     1.     Continued Services and Retirement. Wayland Cato shall continue to serve as an employee, a director and as non-executive Chairman of the Board of the Company to January 31, 2004 (the “Retirement Date”) at which time his resignation as an employee, director and the non-executive chairman of the Board of Directors of the Company shall be effective and he shall retire and relinquish all positions and responsibilities with the Company and its subsidiaries and affiliates (the “Cato Group”) except as specifically set forth herein. Provided, that Wayland Cato shall be entitled, at his option, to resign as a director and/or as the non-executive Chairman of the Board prior to January 31, 2004 without affecting the obligations of the Company set forth herein.

     2.     Continuing Obligations of Wayland Cato. In consideration of the benefits to Wayland Cato set forth in paragraph 3 hereof, Wayland Cato shall comply with the following obligations to the Cato Group following the Retirement Date:

          A. Confidentiality of Company Information. Wayland Cato acknowledges the confidential and proprietary nature of the Confidential Information of the Cato Group and agrees that he will not, without the prior, express written consent of the Chief Executive Officer of the Company, directly or indirectly:

Page 20

Exhibit 10.1             

     For purposes of this Agreement, “Confidential Information” means and includes any and all of the following (whether or not documented):

Page 21

Exhibit 10.1             

Confidential Information shall not include information that is generally known or available to the public or the industry other than as a result of disclosure by Wayland Cato.

          B. Non-solicitation of Cato Group Employees. Wayland Cato agrees that he will not, during the period of time through January 31, 2007, (i) solicit for employment, offer employment to, engage as an employee, independent contractor or in any manner induce or seek to induce any person who is a managerial or administrative employee of the Cato Group to become employed by or engaged as an independent contractor with someone other than the Cato Group, (ii) solicit or encourage any such employee to terminate his or her employment with the Cato Group, or (iii) otherwise interfere with any such employee’s relationship with the Cato Group. Provided, that the foregoing prohibition shall not apply to the employees currently assigned to the non-executive Chairman’s office (namely, Theresa Gebhardt, Sylvia Remeta and Debra Jones) all of whom are anticipated to remain employees of the Company only through December 31, 2003.

          C. Non-Competition. Wayland Cato agrees that for the period of time through January 31, 2007 (the “Restricted Period”) he will not, directly or indirectly, in the Restricted Area (as defined below), Compete (as defined below) with the Cato Group.

          For purposes of this Agreement, the “Restricted Area” means the area within a twenty-five (25) mile radius of any retail store under development, operation or ownership by any member of the Cato Group during the Restricted Period.

          For purposes of this Agreement, “Compete” means to engage in any business activity whatsoever related in any manner or fashion to the operation of retail stores that sell or offer to sell value-priced women’s apparel or accessories. Without limiting the generality of the foregoing, Wayland Cato will not, directly or indirectly (whether for compensation or otherwise), alone or as an agent, principal, partner, officer, employee, trustee, director, shareholder or in any other capacity, own, manage, operate, join, control or participate in the ownership, management, operation or control of, or furnish any capital to, or be connected in any manner with, or provide any services as a consultant for, any business which Competes with the Cato Group in the Restricted Area; provided, however, that notwithstanding the foregoing, nothing contained in this letter shall be deemed (i) to preclude Wayland Cato from owning not more than 5% of the publicly traded securities of any entity which Competes with the Cato Group or (ii) to preclude Wayland Cato from owning or having an ownership interest in shopping centers or buildings which may lease property to businesses that Compete with the Cato Group.

          D. Consulting Services. During the period following the Retirement Date through January 31, 2007, Wayland Cato will be available on reasonable notice and at reasonable times, either in person or by telephone, to furnish to the Cato Group such advisory or consulting services regarding senior management issues as the Company may reasonably call upon Wayland Cato to furnish, provided that Wayland Cato will not be required to devote more than eighty (80) hours of service during any 12 month period to the Cato Group. The Company will reimburse Wayland Cato for any reasonable expenses he incurs in connection with rendering consulting services requested by the Company.

Page 22

Exhibit 10.1             

          E. Office Relocation. No later than the Retirement Date, Wayland Cato will relocate his office from the Company premises. Wayland Cato shall be entitled to remove all or any part of the property listed on Schedule 1 from the Company premises without reimbursement or payment to the Company.

     3.     Obligations of the Company. In consideration of the obligations of Wayland Cato to the Cato Group set forth in paragraph 2 of the Agreement, the Company shall comply with the following obligations to Wayland Cato:

          A. Salary and Benefits. The Company will continue Wayland Cato’s current salary and benefits through January 31, 2004. In the event of Wayland Cato’s death prior to January 31, 2004, the Company will continue to pay his salary to his estate to January 31, 2004.

          B. Continuing Payments. The Company will pay Wayland Cato the sum of $500,000 per year during the three-year period commencing February 1, 2004 in equal monthly installments. In the event of Wayland Cato’s death subsequent to the Effective Date, the Company will pay Wayland Cato’s estate a lump sum amount equal to the then aggregate present value (determined using a discount rate equal to the applicable federal short-term rate in effect under Section 1274(d) of the Internal Revenue Code for the month of his death) of the payments payable pursuant to this subparagraph which remain unpaid to January 31, 2007. Such payment to be made as soon as administratively feasible after his death.

          C. Group Term Life Insurance. For the three year period commencing on the Retirement Date, the Company will continue to provide group term life insurance to Wayland Cato comparable to that provided to him on January 31, 2004.

          D. Split Dollar Life Insurance. The Split-Dollar Life Insurance Agreement dated September 17, 1998 and the Supplemental Compensation Agreement dated September 17, 1998 shall each continue in effect in accordance with their respective terms.

          E. Healthcare. Following the Retirement Date and for the remainder of Wayland Cato’s life, the Company will provide Wayland Cato and his wife, Marion Cato, with the same or comparable group healthcare coverage on the same economic terms and conditions as it provides such coverage to its senior executives. In the event Wayland Cato predeceases Marion Cato, the Company will provide such coverage to her for her life (if she is married to Wayland Cato at the time of his death).

          F. Transition Expenses. As soon as is administratively feasible after the Effective Date, the Company will pay Wayland Cato the sum of $100,000 to assist with the cost of the relocation of his office.

          The Company shall have no obligations to Wayland Cato following the Effective Date except as is specifically set forth in the Agreement; provided Wayland Cato shall continue

Page 23

Exhibit 10.1             

to be entitled to any benefits due to him under the benefit plans listed on Schedule 2 attached hereto.

     4.     Tax. Wayland Cato acknowledges that income may be imputed to him in connection with the receipt of the benefits provided under the Agreement and that Wayland Cato will be responsible for the payment of all federal or state tax liabilities, penalties, interest, tax payments or tax judgments that could arise as a result of these benefits; provided, that this shall not affect the obligations of the Company under the Supplemental Compensation Agreement referred to in subparagraph 3.D. above. The Company may withhold from any amounts or benefits payable or provided under this Agreement such federal, state and local taxes as are required to be withheld pursuant to any applicable law or regulation.

     5.     Acknowledgements. Wayland Cato acknowledges that he has carefully read this Agreement, that he knows and understands the contents of this Agreement, that he has consulted with a lawyer regarding this Agreement, and that he executes this Agreement of his own free will. Wayland Cato and the Company agree that the mutual consideration to the parties hereto is fair and reasonable.

     6.     Governing Law. This Agreement shall be governed by and construed in accordance with the laws of the State of North Carolina.

     7.     Notices. Any notice or other communications to be given hereunder shall be deemed to have been given or delivered when delivered by hand to the individuals named below or when deposited in the United States mail, registered or certified, with proper postage and registration or certification fees prepaid, addressed to the parties as follows (or to such other address as one party shall give the other in the manner provided herein):

     8.     Entire Agreement. This Agreement constitutes the entire agreement between the parties pertaining to the subject matter contained herein and supersedes any and all prior and contemporaneous agreements, representations, promises, inducements and understandings of the parties. This written Agreement cannot be varied, contradicted or supplemented by evidence of any prior or contemporaneous oral or written agreements. Moreover, this written Agreement may not be later modified except by a further writing signed by a duly authorized officer of the Company and Wayland Cato.

Page 24

Exhibit 10.1             

     IN WITNESS WHEREOF, the parties have executed this Agreement as of the day and year first above written.

Page 25

Exhibit 10.1             

Schedule 1

Office furniture and filing cabinets currently located in the Chairman’s office complex

1999 Ford Taurus

Oil portrait of Wayland H. Cato, Sr. which is currently in storage

Portraits of individual directors and directors as a group currently located in the Board Room at such time as the Company decides not to display such portraits

Page 26

Exhibit 10.1             

Schedule 2

1.     The Cato Corporation 401(k) Profit Sharing Plan

2.     The Cato Corporation Life and AD&D Plan – Option to convert within 31 days following retirement