FEDERAL TRUST BANK EMPLOYEE SEVERANCE AGREEMENT

EXHIBIT 10.6

FEDERAL TRUST BANK

EMPLOYEE SEVERANCE AGREEMENT

THIS EMPLOYEE SEVERANCE AGREEMENT (“Agreement”) is entered into by and between Federal Trust Bank (“Employer”) and Gregory E. Smith (“Employee”).

WHEREAS, in recognition of Employee’s contribution to Employer as its Executive Vice President & Chief Financial Officer. Employer wishes to protect Employee’s position therewith in the manner provided in this Agreement in the event of a Change in Control of the Employer (as defined below), or of its parent-holding company, Federal Trust Corporation (“FTC”);

NOW, THEREFORE, in consideration of Employee’s management position, contribution, and responsibilities, Employer hereby agrees to provide Employee with certain severance benefits as specifically provided herein.

SECTION 1 - DEFINITIONS

(a) “Change in Control” means the sale, dissolution, merger, consolidation or other reorganization of the Employer or FTC, any sale of all or substantially all of the assets of the Employer or FTC, or any sale or other transfer of the ownership of, or the right to vote, more than 50% of the total combined voting power of all classes of stock entitled to vote on the election of directors (“voting stock”) of the Employer or FTC; provided, however, a “Change in Control” shall be deemed not to have occurred in connection with any transaction where stockholders who own more than 50% of the voting stock of the Employer or FTC immediately before the consummation of such transaction, own more than 50% of the voting stock or other controlling ownership interests of the surviving entity immediately after such transaction.

(b) Termination for “Just Cause” means termination of Employee’s employment by the Employer, upon written notice to Employee, upon any of the following events:

(i) Employee’s continued willful failure to perform, or his habitual neglect of, his duties;

(ii) Employee’s conviction of, plea of nolo contendre to, indictment or information for (which indictment or information is not discharged or otherwise resolved within eighteen (18) months), any felony, or any crime involving moral turpitude, or any crime which is likely to result in material injury to the Employer;

(iii) Employee’s commission of an act of fraud, theft or dishonesty, or breach of a fiduciary duty relating to the Employee’s employment with the Employer;

(iv) Employee’s material breach of this Agreement;

(v) Employee’s having engaged in conduct involving fraud, deceit, personal dishonesty, or breach of fiduciary duty;

(vi) Employee’s having violated any banking law or regulation, memorandum of understanding, cease and desist order, or other agreement with any banking agency having jurisdiction over FTC or the Employer;

(vii) Employee’s having become subject to continuing intemperance in the use of alcohol or drugs which has adversely affected, or may adversely affect, the business or reputation of FTC or the Employer, as determined by the Board of Director of the Employer (without taking into consideration the vote of Employee as a director) in its sole and absolute discretion; or

(viii) Any banking authority having supervisory jurisdiction over FTC or the Employer initiating any proceeding for removal of the Employee.

Notwithstanding the foregoing, Employee shall not be deemed to have been terminated for Just Cause under clause (i) or (iv) above unless the Employer provided reasonable written notice to the Employee setting forth the reasons for the Employer’s intention to terminate for Just Cause, and Employee failed within thirty (30) days to cure the event or deficiency set forth in the written notice.

(c) “Protected Period” means the term of this Agreement and (3) Three months following termination hereof.

SECTION 2 - TERM OF AGREEMENT

This Agreement shall remain in effect during the term of Employee’s employment by Employer, unless and until Employer, at any time and in its sole and absolute discretion, shall earlier terminate this Agreement by delivering to the Employee written notice indicating that this Agreement is terminated.

SECTION 3 - PAYMENTS TO EMPLOYEE UPON CHANGE IN CONTROL

In the event of a Change in Control within the Protected Period, if either (i) Employer terminates Employee’s employment without Just Cause, or (ii) Employee terminates his own employment for any reason, whether such termination occurs prior to or after the Change of Control, Employee shall be entitled to receive the termination benefits described in Section 4. This Section 3 shall survive termination of this Agreement for the Protected Period.

SECTION 4 - TERMINATION BENEFITS

(a) Upon a termination of employment described in Section 3, Employer shall pay Employee, or in the event of his subsequent death, his estate, as severance pay, a sum equal to two years’ “highest annual base salary.” For purposes of this Agreement, Employee’s highest annual base salary shall mean the Employee’s highest base salary during the three years immediately preceding Employee’s termination. Such payment shall be made in one lump sum payment six (6) months after a termination of employment.

(b) Upon a termination described in Section 3, Employer or its successor(s) shall continue to provide life, health and disability coverage (“Coverage”) comparable to the coverage maintained by Employer for Employee prior to his termination. Such Coverage shall cease upon

2

the earlier of Employee’s obtaining new employment and receiving similar Coverage through another employer, which provides comparable coverage, or one year from the date of Employee’s termination.

(c) Notwithstanding the foregoing, the Employee shall not receive any termination benefits described in this Section 4 unless and until the Employee has executed and delivered to the Employer a full and unconditional release, in a form reasonably acceptable to the Employer, of all claims which the Employee may have against the Employer or FTC in connection with the Employee’s employment or the termination thereof, other than (i) the Employee’s claim for the termination benefits described in this Section 4, and (ii) any claim which cannot be waived by law.

SECTION 5 - SUSPENSION OF OBLIGATIONS

(a) If Employee is suspended from office and/or temporarily prohibited from participating in the conduct of Employer’s affairs pursuant to an action brought by the Office of Thrift Supervision or the Federal Deposit Insurance Corporation (either referred to herein as a “Regulatory Agency”), Employer’s obligations under this Agreement shall be suspended as of the date of such action. The obligations of this Agreement shall be reinstated if the charges of the Regulatory Agency are subsequently dismissed, or if the Employee is otherwise determined to be not guilty of such charges.

(b) If Employee is removed from office or permanently prohibited from participating in Employer’s conduct or affairs by a final order resulting from an action brought by a Regulatory Agency, all obligations of Employer under this Agreement shall terminate on the effective date of the order.

SECTION 6 - NOTICE OF TERMINATION

Any purported termination of employment by Employer or by Employee shall be communicated by a Notice of Termination to the other party hereto. For purposes of this Agreement, a “Notice of Termination” shall mean a written notice which shall set forth in reasonable detail the facts and circumstances claimed to provide a basis for termination of Employee’s employment under the provision so indicated.

SECTION 7 - NON-SOLICITATION; CONFIDENTIALITY

In consideration of the benefits and protections of this Agreement, Employee agrees to the following:

(a) During the term of Employee’s employment by the Employer, and for a period of one year following the termination of Employee’s employment for any reason, Employee will not, directly or indirectly:

(i) solicit, divert, or take away, or attempt to solicit, divert, or take away from the Employer the business of any person who the Employee knows or reasonably should know is a customer or identified, prospective customer of the Employer, or cause any such person to refrain, in any respect, from conducting business with the Employer; or

3

(ii) solicit, request, or induce, or attempt to solicit, request, or induce any employee of the Employer or FTC to terminate employment with the Employer or FTC and accept employment with another person, firm, corporation, or other entity; provided, however, that general advertisements for employment that are not directed to the Employer’s or FTC’s employees will not violate this Section 7.

(b) The Employee will not at any time use for his own benefit, copy or make known in any manner to any person, firm, corporation or other entity the contents of any agreements (including this Agreement), memoranda, correspondence, writings, drawings, reports, charts, or other media, of or related to information, data, methods, systems, processes, concepts or technologies, used or developed by the Employee, the Employer, or FTC, including, without limitation, any and all trade secrets (as defined under Florida law), proprietary information or other confidential information acquired by the Employee in connection with the Employee’s employment with the Employer. The Employee understands and agrees that the lists of existing or prospective customers, vendors, and contractors of the Employer, as such may exist from time to time, and information concerning such customers, vendors, and contractors are valuable, special and unique assets of the Employer’s business which are entitled to protection under the provisions of this Section 7.

(c) The parties acknowledge and agree that money damages cannot fully compensate Employer in the event of Employee’s violation of the provisions of this Section 7. Thus, in the event of a breach of any of the provisions of this Section 7, Employee agrees that Employer, upon application to a court of competent jurisdiction, shall be entitled to an injunction restraining Employee from any further breach of the terms of this Section 7. Employee’s sole remedy, in the event of the wrongful entry of such injunction, shall be the dissolution of such injunction. Employee hereby waives any and all claims for damages by reason of the wrongful issuance of any such injunction.

(d) This Section 7 shall survive termination of this Agreement for any reason.

SECTION 8 - ENFORCEMENT COSTS

The Employer is aware that upon the occurrence of a Change in Control, the Board of Directors or a stockholder of the Employer may then cause or attempt to cause the Employer to refuse to comply with its obligations under this Agreement, or may cause or attempt to cause the Employer to institute, or may institute, litigation seeking to have this Agreement declared unenforceable, or may take, or attempt to take, other action to deny the Employee the benefits intended under this Agreement. In these circumstances, the purpose of this Agreement could be frustrated. It is the intent of the parties that the Employee not be required to incur the legal fees and expenses associated with the protection or enforcement of his rights under this Agreement in the event of a Change of Control by litigation or other legal action because such costs would substantially detract from the benefits intended to be extended to the Employee hereunder, nor be bound to negotiate any settlement of his rights hereunder under threat of incurring such costs in the event of a Change of Control. Accordingly, if in connection with, or as a result of, a Change of Control, it should reasonably appear to the Employee that the Employer is or has acted contrary to or is failing or has failed to comply with any of its obligations under this Agreement for the reason that it regards this Agreement to be void or unenforceable or for any other reason, or in the event that the Employer or any other person takes any action to declare this Agreement

4

void or unenforceable in connection with, or as a result of, a Change of Control, or institutes any litigation or other legal action designed to deny, diminish or to recover from the Employee the benefits provided or intended to be provided to him hereunder in connection with, or as a result of, a Change of Control, and the Employee has acted in good faith to perform his obligations under this Agreement, the Employer irrevocably authorizes the Employee from time to time to retain counsel of his choice at the expense of the Employer (as provided below) to represent him in connection with the protection and enforcement of his rights hereunder arising from such Change of Control, including without limitation representation in connection with the initiation or defense of any litigation or other legal action, whether by or against the Employee or the Employer or any director, officer, stockholder or other person affiliated with the Employer, in any jurisdiction. The reasonable fees and expenses of counsel selected from time to time by the Employee as herein above provided shall be paid or reimbursed to the Employee by the Employer on a regular, periodic basis upon presentation by the Employee of a statement or statements prepared by such counsel in accordance with its customary practices. Counsel so retained by the Employee may be counsel representing other officers or key executives of the Employer in connection with the protection and enforcement of their rights under similar agreements between them and the Employer, and, unless in his sole judgment use of common counsel could be prejudicial to him or would not be likely to reduce the fees and expenses chargeable hereunder to the Employer, the Employee agrees to use his best efforts to agree with such other officers or executives to retain common counsel.

SECTION 9 - MODIFICATION AND WAIVER

(a) This Agreement may not be modified or amended except as agreed to in writing by the parties hereto. Notwithstanding the foregoing, this Section 8(a) shall not be construed to limit the Employer’s ability under Section 2 to terminate this Agreement.

(b) No term or condition of this Agreement shall be deemed to have been waived, nor shall there be any estoppel against the enforcement of any provision of this Agreement, except by written instrument of the party charged with such waiver or estoppel. No such written waiver shall be deemed a continuing waiver unless specifically stated therein, and each such waiver shall operate only as to the specific term or condition waived and shall not constitute a waiver of such term or condition for the future, or as to any act other than that specifically waived.

SECTION 10 - ARBITRATION

The parties agree that, except for the specific remedies for injunctive relief as contained in Section 7, any controversy or claim arising out of or relating to this Agreement or any breach hereof, including, without limitation, any claim that this Agreement or any portion hereof is invalid, illegal or otherwise voidable, shall be submitted to binding arbitration before and in accordance with the rules of the American Arbitration Association, and judgment upon the determination and/or award of such arbitrator(s) may be entered in any court having jurisdiction thereof. This Section shall not be construed to permit the award of punitive damages to either party. The venue of any arbitration shall be in Seminole County, Florida.

5

SECTION 11 - ATTORNEYS’ FEES

In the event of any proceeding occurring out of or involving this Agreement, the prevailing party shall be entitled to recovery of reasonable attorneys’ fees, expenses, and costs, including fees and costs to enforce an award.

SECTION 12 - SEVERABILITY

The invalidity or unenforceability of any provision of this Agreement shall not affect the validity or enforceability of any other provision of this Agreement, which shall remain in full force and effect.

SECTION 13 - HEADINGS FOR REFERENCE ONLY

The headings of the sections herein are included solely for convenience of reference and shall not control the meaning or the interpretation of any of the provisions of this Agreement.

SECTION 14 - APPLICABLE LAW

This Agreement shall be governed in all respects and be interpreted by and under the laws of the State of Florida.

SECTION 15 - SUCCESSORS

Employer shall require any successor to the business of Employer in connection with a Change in Control to assume and agree to perform Employer’s obligations under this Agreement in writing.

SECTION 16 - NO CONTRACT OF EMPLOYMENT

This Agreement shall not, under any circumstances, be deemed to constitute an employment contract between Employer and Employee or to be in consideration of or an inducement for the employment of Employee. Nothing contained in this Agreement shall be deemed to give Employee the right to be retained in the service of Employer, or to interfere with the right of Employer to discharge Employee at any time.

SECTION 17 - LIMITATION OF RIGHTS

Neither this Agreement, nor any amendment hereof, nor the payment of any benefits hereunder shall be construed as giving Employee or other person any legal or equitable right against Employer except as expressly herein.

6

IN WITNESS WHEREOF, Employee and Employer have duly executed this Agreement this 5th day of January, 2008

7