The recently proposed nonlinear ensemble transform filter (NETF) is extended to a fixed lag smoother. The NETF approximates Bayes' equation by applying a square root update based on weights computed from a particle filter. As an ensemble transform filter the NETF shares similarities with the widely used ETKF and can be localized analogously. Further, the smoother extension NETS can by obtained by applying the transform matrix for filtering to the ensembles at previous analysis times. To assess the nonlinear assimilation method in a high-dimensional test case, the effectiveness of the nonlinear filter and the new smoother is assessed by twin experiments with a square box configuration of NEMO ocean model. The results show that the NETF reaches a comparable assimilation performance as the LETKF. The smoothing in the NETS effectively reduces the errors in the state estimates. Different variables show very similar optimal smoothing lags, which allows for a simultaneous tuning of the lag to obtain minimal smoothing errors. In comparison to the LESTKS, the NETS is slightly less effective and the optimal lag in the NETS is shorter. This difference is caused by the different update mechanisms of both filters and can depend on the nonlinearity of the model.