How should I tokenize a string into char using boost

Question

I am trying to tokenize a string into character using boost

The present boost tokenizer will tokenize based on space

 typedef boost::tokenizer<boost::char_separator<char> >
    tokenizer;
  boost::char_separator<char> sep("");
  tokenizer tokens(str, sep);

I expect output to be j e f but the the actual output is jef

Answer 1

This

""

is string literal with no characters ended by null terminator. Whereas

" "

is string literal which contains one character - space also ended by null terminator. If you want to split str = "j e f" by space you need to write something like this:

  typedef boost::tokenizer<boost::char_separator<char> >
        tokenizer;
  boost::char_separator<char> sep(" ");
  std::string str = "j e f";
  tokenizer tokens(str, sep);
  for (auto i : tokens)
    cout << i << endl;
  // output
  j
  e
  f

As the name char_separator suggests it takes characters, your string "" contains no chars. Splitting is realized by comparing separator character with input string. How do you want to do this comparison when there is no character to do it, i.e. ""?