PHP is letting you know that the maximum amount of memory that it is allowed to use has been exhausted. You need to either do less memory intensive things (work with smaller images, etc), or increase the maximum amount of memory PHP is allowed to use on your server.
When working with really large images, or even doing processor intensive filters on moderately large images, you'll need to increase the maximum amount of memory that PHP can access on your server. You can easily do so in php.ini or right in your EE config.php file like so:
$config['ce_image_memory_limit'] = 128;
The default limit is 64 (megabytes), so you can try bumping it up a bit. See the CE Image Basic Configuration page for more details on the config setting.
Another thing to keep in mind, is that CE Image is not the only thing that is using memory. The memory usage is comprised of all resources that PHP has in memory: files, snippets, add-on classes, etc. Image manipulation is innately memory heavy though.
Disclaimer: I'm the author of CE Image.