BURLINGTON COAT FACTORY HOLDINGS, INC. NON-COMPETITION AND CONFIDENTIALITY AGREEMENT

Exhibit 2.4

BURLINGTON COAT FACTORY HOLDINGS, INC.

NON-COMPETITION AND CONFIDENTIALITY AGREEMENT

THIS AGREEMENT is made as of April 13, 2006 between Burlington Coat Factory Holdings, Inc., a Delaware corporation (the “Company”), and Stephen Milstein (“Seller”). For purposes of this Agreement, unless the context requires otherwise, the term “Company” shall include all subsidiaries of the Company, including, without limitation, Burlington Coat Factory Warehouse Corporation, a Delaware corporation (“BlueBlazer”) and its subsidiaries.

WHEREAS, Seller pursuant to that certain Agreement and Plan of Merger by and among the Company, BCFWC Mergersub, Inc., a Delaware corporation (the “Merger Sub”) and BlueBlazer, dated as of January 18, 2006 (the “Merger Agreement”) has received from the Company the right to receive certain cash consideration in exchange for certain securities of BlueBlazer.

WHEREAS, at the Effective Time (as defined in the Merger Agreement) MergerSub will merge with and into BlueBlazer pursuant to the Merger Agreement, with BlueBlazer being the surviving corporation of such merger, and the Company will own all issued and outstanding shares of capital stock of BlueBlazer.

WHEREAS, the Company and Seller desire to enter into an agreement setting forth the obligation of Seller to refrain from competing with the Company for a period of time after the consummation of the transactions contemplated by the Merger Agreement as provided herein.

WHEREAS, the execution and delivery of this Agreement by Seller is a condition to the consummation of the transactions contemplated by the Merger Agreement, and Seller is entering into this Agreement to induce the Company and the Merger Sub to consummate the transactions contemplated by the Merger Agreement.

NOW, THEREFORE, in consideration of the mutual covenants contained herein and other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the Company and Seller hereby agree as follows:

1. Nondisclosure and Nonuse of Confidential Information. For a period of three (3) years following the date hereof, Seller shall keep secret and hold in confidence, and shall not use for its benefit, any and all information relating to the Company that is proprietary to the Company, other than the following: (i) information that has become generally available to the public other than as a result of a disclosure by Seller in breach of this Agreement and (ii) information that is required to be disclosed by any applicable law. In connection with disclosure of confidential information under clause (ii) above, Seller shall give the Company timely prior notice of the anticipated disclosure and the parties shall cooperate in designing reasonable procedural and other safeguards to preserve, to the maximum extent possible, the confidentiality of such material.

2. Noncompetition.

(a) Seller agrees that, for a period of one (1) year following the date hereof (the “Non-Compete Period”), neither Seller nor any of Seller’s Affiliates shall without the prior written consent of the Company, directly or indirectly, anywhere in the United States (the “Territory”) (i) form, acquire, operate, control, make a financial investment in, enter into any agreement pertaining to, publicly announce the launch of, or otherwise finance, manage, participate in, consult with, become employed by or render advisory services to, any business or otherwise become associated with an enterprise, the business of which is the same as, substantially similar to or otherwise competitive with the business of the Company as now conducted (i.e., the business of BlueBlazer and its subsidiaries conducted or proposed to be conducted immediately prior to the closing of the Merger) (a “Competing Business”), or (ii) for the purpose of conducting or engaging in a Competing Business, call upon, solicit, advise or otherwise do, or attempt to do, business with any suppliers, vendors or other material business relationships of the Company. As used in this Section 2 and Section 3 below, “Affiliate” of any particular person means any other person controlling, controlled by or under common control with such particular person, where “control” means the possession, directly or indirectly, of the power to direct the management and policies of a person whether through the ownership of voting securities, contract or otherwise. Nothing herein shall prohibit Seller or any of Seller’s Affiliates from being a passive owner of not more than 3% of the outstanding stock of a corporation which is publicly traded, so long as neither Seller nor any of Seller’s Affiliate have an active participation in the business of such corporation.

(b) Seller agrees that (i) the covenants set forth in this Section 2 are reasonable in temporal and geographical scope and in all other respects, and (ii) the covenants contained herein have been made in order to induce the Company to enter into this Agreement and the Merger Agreement.

(c) The Company and Seller intend that the covenants of this Section 2 shall be deemed to be a series of separate covenants, one for each county or province of each and every state within the Territory and one for each month of the Non-Compete Period.

(d) If, at the time of enforcement of this Section 2, a court shall hold that the duration or scope stated herein are unreasonable under circumstances then existing, the parties agree that the maximum duration or scope under such circumstances shall be substituted for the stated duration or scope and that the court shall be allowed to revise the restrictions contained herein to cover the maximum period and scope permitted by law.

(e) Seller recognizes and affirms that in the event of its breach of any provision of this Section 2, money damages would be inadequate and the Company would not have an adequate remedy at law. Accordingly, Seller agrees that in the event of a breach or a threatened breach by Seller of any of the provisions of this Section 2, the Company, in addition and supplementary to other rights and remedies existing in its favor, may apply to any court of law or equity of competent jurisdiction for specific performance and/or injunctive or other relief in order to enforce or prevent any violations of the provisions hereof (without posting a bond or other security). In addition, in the event of a breach or violation by Seller of this Section 2, the Non-Compete Period shall be tolled until such breach or violation has been duly cured.

2

3. Nonsolicitation. For a period of three (3) years following the date hereof, Seller and each of Seller’s Affiliates shall not:

(a) induce or attempt to induce any employee of the Company to leave the employ of the Company, or in any way interfere with the relationship between the Company and any employee thereof; provided that the placement of any general advertisement or general solicitation to the public at large shall not constitute “interference” under this clause (a);

(b) hire directly or through another entity any person who was an employee of the Company or any of its subsidiaries as of the date hereof; provided that, with the Company’s prior written consent (which consent shall not unreasonably be withheld), Seller and Seller’s Affiliates can hire any person who was an employee of the Company or any of its subsidiaries who has been fired by the Company or any of its subsidiaries; provided further that

(i) for the avoidance of doubt, the Company’s withholding of consent with respect to Seller’s or Seller’s Affiliate’s hiring of any person shall be deemed reasonable if the Company determines in good faith that (x) the hiring of such person by Seller or Seller’s Affiliates would be detrimental to the business of the Company or (y) such person has caused such firing to occur in order to circumvent the restrictions of this Section 3(b),

(ii) if Seller or any of Seller’s Affiliates wishes to hire any such person or persons, it shall so notify the Company (the “Notice’), which Notice shall set forth the identity of the former employee or employees and request the consent of the Company to such hiring, and, if the Company fails to notify the Seller or the notifying Affiliate within ten (10) days of the Company’s receipt of the Notice to it that it is withholding its consent, the Company shall be deemed to have granted such consent and the Seller or the notifying Affiliate may proceed to hire such former employee or employees; and

(iii) nothing herein shall prevent a member of the Milstein family from hiring any other member of the Milstein family; or

(c) induce or attempt to induce any supplier, vendor or other business relation of the Company to cease doing business with the Company, or in any way interfere with the relationship between any such supplier, vendor, or other business relation and the Company (including, without limitation, making any negative statements or communications concerning the Company); provided that (x) the hiring of any legal counsel, accountant or financial advisor, and (y) following the end of the Non-Compete Period, the operation of any competitive business in the ordinary course of business, in each case, in and of itself, shall not constitute “interference” under this Section 3(c).

4. Notices. Any notice provided for in this Agreement must be in writing and must be either personally delivered, sent via facsimile, mailed by first class mail (postage prepaid and return receipt requested) or sent by reputable overnight courier service (charges prepaid) to the recipient at the address below indicated:

To the Company:

Burlington Coat Factory Holdings, Inc.

Bain Capital Partners, LLC

111 Huntington Avenue

Boston, Massachusetts 02199

Attention: Jordan Hitch

Facsimile No.: (617)  ###-###-####

3

with copies (which shall not constitute notice to the Company) to:

Kirkland & Ellis

Citicorp Center

153 East 53rd Street

New York, NY 10022-4675

Attention: Lance Balk, Esq.

Facsimile: (212)  ###-###-####

To Seller:

Stephen Milstein

1005 Plowshare Road

Yardley, Pennsylvania 19067

with copies (which shall not constitute notice to the Seller) to:

Phillips Nizer LLP

666 Fifth Avenue

28^th Floor

New York, NY 10103

Attention: Tiberio Schwartz

Facsimile: (212)  ###-###-####

or such other address or to the attention of such other person as the recipient party shall have specified by prior written notice to the sending party. Any notice under this Agreement shall be deemed to have been given when received by the intended recipient thereof.

5. General Provisions.

(a) Absence of Conflicting Agreements. Seller hereby warrants and covenants that (i) Seller’s execution, delivery and performance of this Agreement do not and shall not result in a breach of the terms, conditions or provisions of any agreement, instrument, order, judgment or decree to which Seller is subject and (ii) upon the execution and delivery of this Agreement by the Company, this Agreement shall be the valid and binding obligation of Seller, enforceable in accordance with its terms.

(b) Severability. Whenever possible, each provision of this Agreement shall be interpreted in such manner as to be effective and valid under applicable law, but if any

4

provision of this Agreement is held to be invalid, illegal or unenforceable in any respect under any applicable law or rule in any jurisdiction, such invalidity, illegality or unenforceability shall not affect any other provision or any other jurisdiction, and this Agreement shall be reformed, construed and enforced in such jurisdiction as if such invalid, illegal or unenforceable provision had never been contained herein. The parties agree that a court of competent jurisdiction making a determination of the invalidity or unenforceability of any term or provision of Section 2 of this Agreement shall have the power to reduce the scope, duration or area of any such term or provision, to delete specific words or phrases or to replace any invalid or unenforceable term or provision in Section 2 with a term or provision that is valid and enforceable and that comes closest to expressing the intention of the invalid or unenforceable term or provision, and this Agreement shall be enforceable as so modified.

(c) Complete Agreement. This Agreement, those documents expressly referred to herein and other documents of even date herewith embody the complete agreement and understanding among the parties and supersede and preempt any prior understandings, agreements or representations by or among the parties, written or oral, which may have related to the subject matter hereof in any way.

(d) Counterparts. This Agreement may be executed in separate counterparts, each of which is deemed to be an original and all of which taken together constitute one and the same agreement.

(e) Successors and Assigns. Except as otherwise provided herein, this Agreement shall bind and inure to the benefit of and be enforceable by the Company and Seller and their respective successors and assigns; provided, that the rights and obligations of Seller under this Agreement may not be assigned (except in connection with a liquidation or dissolution of Seller) or delegated without the prior written consent of the Company.

(f) Choice of Law. All questions concerning the construction, validity, enforcement and interpretation of this Agreement shall be governed by, and construed in accordance with, the internal law, and not the law of conflicts, of the State of New York, without giving effect to any choice of law or conflict of law rules or provisions (whether of the State of New York or any other jurisdiction) that would cause the application of the laws of any jurisdiction other than the State of New York.

(g) Waiver of Jury Trial. EACH OF THE COMPANY AND SELLER HEREBY IRREVOCABLY WAIVES ANY AND ALL RIGHT TO TRIAL BY JURY IN ANY ACTION OR PROCEEDING ARISING OUT OF OR RELATED TO THIS AGREEMENT OR THE TRANSACTIONS CONTEMPLATED BY THE MERGER AGREEMENT.

(h) Jurisdiction. Each of the parties hereto submits to the jurisdiction of any state or federal court sitting in New York, New York, in any action or proceeding arising out of or relating to this Agreement and agrees that all claims in respect of the action or proceeding may be heard and determined in any such court and hereby expressly submits to the personal jurisdiction and venue of such court for the purposes hereof and expressly waives any claim of improper venue and any claim that such courts are an inconvenient forum. Each of the parties

5

hereby irrevocably consent to the service of process of any of the aforementioned courts in any such suit, action or proceeding by the mailing of copies thereof by registered or certified mail, postage prepaid and return receipt requested, to its address set forth in Section 4, such service to become effective 10 days after such mailing.

(i) Descriptive Headings; Construction. The descriptive headings of this Agreement are inserted for convenience only and do not constitute a part of this Agreement. The parties to this Agreement have participated jointly in the negotiation and drafting of this Agreement. In the event an ambiguity or question of intent or interpretation arises, this Agreement shall be construed as if drafted jointly by the parties and no presumption or burden of proof shall arise favoring or disfavoring any party by virtue of the authorship of any of the provisions of this Agreement. The word “including” shall mean including without limitation.

(j) No Third-Party Beneficiaries. This Agreement is for the sole benefit of the parties hereto and their permitted successors and assigns and nothing herein expressed or implied shall give or be construed to give any person, other than the parties hereto and such permitted successors and assigns, any legal or equitable rights hereunder.

(k) Remedies. Each of the parties to this Agreement shall be entitled to enforce its rights under this Agreement specifically, to recover damages and costs (including reasonable attorneys fees) caused by any breach of any provision of this Agreement and to exercise all other rights existing in its favor. The parties hereto agree and acknowledge that Seller’s breach of any term or provision of this Agreement shall materially and irreparably harm the Company, that money damages shall accordingly not be an adequate remedy for any breach of the provisions of this Agreement by Seller and that the Company in its sole discretion and in addition to any other remedies it may have at law or in equity may apply to any court of law or equity of competent jurisdiction (without posting any bond or deposit) for specific performance and/or other injunctive relief in order to enforce or prevent any violations of the provisions of this Agreement. Nothing herein is intended to entitle any party hereto to money damages without proving such damages.

(l) Amendment and Waiver. The provisions of this Agreement may be amended and waived only with the prior written consent of the Company and Seller.

* * * * *

6

IN WITNESS WHEREOF, the parties hereto have executed this Non-Competition and Confidentiality Agreement on the date first written above.