Using the De Bruijn notation, it is possible to define lambda terms as:
data BTerm = BVar Int | BLam BTerm | BApp BTerm BTerm
Or using the usual notation,
data Term = Var String | Lam String Term | App Term Term
These two datatypes allow construction of both closed terms and terms containing free variables.
Is it possible to define a datatype that allows construction of only closed terms. ie only the terms such as: \x.x, \x. x x, \x.\y. x y, \x.\y. y, \x.\y.\z.z(x y)
If you want to encode it in some form that looks close to arbitrary lambda expressions, I suspect it will require some type-level programming to track the current lambda depth. This will make it hard to combine terms without knowing at compile time what types those terms have.
But if you don't mind something equivalent that looks quite different, it is well known that SKI combinators are equivalent to lambda calculus. And since SKI offers no explicit lambdas or indeed variable references at all, there's no way to encode a non-closed term.