"Brad" wrote: > Is there a way to determine whether a file is plain ascii text or not > using standard C++?
No. It's in the eye of the beholder. You can make a very good guess by looking by counting control characters that wouldn't likely be in text. But the possibility exists that a binary file might not have any of them either.
Medvedev wrote: > On Jul 5, 11:22 am, Sherman Pendley <spamt...@dot-app.org> wrote: >> Brad <b...@16systems.com> writes: >>> Is there a way to determine whether a file is plain ascii text or not >>> using standard C++? >> Sure, just read its contents and look for any byte that's > 127. If >> you find one, the file's contents are not plain ASCII.
> if he try to test in a text file which contain non-English text , he > will fail!! > because non-English char are > 127
On Jul 5, 11:45 am, Medvedev <3D.v.Wo...@gmail.com> wrote:
> On Jul 5, 11:22 am, Sherman Pendley <spamt...@dot-app.org> wrote:
> > Brad <b...@16systems.com> writes: > > > Is there a way to determine whether a file is plain ascii text or not > > > using standard C++?
> > Sure, just read its contents and look for any byte that's > 127. If > > you find one, the file's contents are not plain ASCII.
> if he try to test in a text file which contain non-English text , he > will fail!! > because non-English char are > 127
sorry man , u r right i found non-English represented by negative sign and binary is the file which it's byte MAY BE > 127 as it can hold 256-bit pattern
Medvedev <3D.v.Wo...@gmail.com> writes: > On Jul 5, 11:22 am, Sherman Pendley <spamt...@dot-app.org> wrote: >> Brad <b...@16systems.com> writes: >> > Is there a way to determine whether a file is plain ascii text or not >> > using standard C++?
>> Sure, just read its contents and look for any byte that's > 127. If >> you find one, the file's contents are not plain ASCII.
> if he try to test in a text file which contain non-English text , he > will fail!!
On Jul 5, 9:45 pm, Medvedev <3D.v.Wo...@gmail.com> wrote:
> On Jul 5, 11:22 am, Sherman Pendley <spamt...@dot-app.org> wrote: > > Brad <b...@16systems.com> writes: > > > Is there a way to determine whether a file is plain ascii text or not > > > using standard C++? > > Sure, just read its contents and look for any byte that's > 127. If > > you find one, the file's contents are not plain ASCII. > if he try to test in a text file which contain non-English > text , he will fail!! because non-English char are > 127
ASCII is a seven bit code, so no characters are greater than 127 in it.
Of course, just because you don't find any characters greater than 127 doesn't mean that it is ASCII. It could still be ISO 8859-1, or UTF-8, in which, by chance, none of the characters happen to be greater than 127. (Or it could be that plain char is signed on your machine, in which case, it can't contain a value greater that 127, regardless of the encoding:-).)
-- James Kanze (GABI Software) email:james.ka...@gmail.com Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Stefan Ram wrote: > Brad <b...@16systems.com> writes: >> Is there a way to determine whether a file is plain ascii text >> or not using standard C++?
> If someone can define in words when a file is deemed to be a > »a plain ascii text« without ambiguity and for each possible > file, I am sure that then this newsgroup will be able to > help to implement a test for it in C++.
> ...
Thanks for all the responses. The program recurses through a directory processing files. I do not know beforehand what type of files the program may encounter. The processing is simply reading the file and passing its content to a regular expression to search for certain strings.
Binary files cause problems, so I thought if I could just skip them and only read ASCII and perhaps UTF-8 encoded files, things would be better. That lead to my initial question. Later I could learn how to deal with binary files that I may want to search like PDF and MS Office documents. Just curious if standard C++ had some built-in function that made this easy.
Brad writes: > That lead to my initial question. Later I could learn how to deal with > binary files that I may want to search like PDF and MS Office documents. > Just curious if standard C++ had some built-in function that made this easy.
No. The only 'built-in' function of any kind is one to test if a single character belongs in a given character class: isascii() and its equivalents. It's up to you to scan the entire contents of the file, to classify it.
In POSIX, you might be able to get away with opening a file, stat()ing its contents, to get the file's size, mmap-ing the file into memory, then using std::find_if() to search for non-ascii bytes. Of course, if you hit a 4gb file, that might cause ...problems.
> Stefan Ram wrote: >> Brad <b...@16systems.com> writes: >>> Is there a way to determine whether a file is plain ascii text >>> or not using standard C++?
>> If someone can define in words when a file is deemed to be a >> »a plain ascii text« without ambiguity and for each possible >> file, I am sure that then this newsgroup will be able to >> help to implement a test for it in C++. > > ...
> Thanks for all the responses. The program recurses through a directory > processing files. I do not know beforehand what type of files the > program may encounter. The processing is simply reading the file and > passing its content to a regular expression to search for certain strings.
> Binary files cause problems, so I thought if I could just skip them and > only read ASCII and perhaps UTF-8 encoded files, things would be better. > That lead to my initial question. Later I could learn how to deal with > binary files that I may want to search like PDF and MS Office documents. > Just curious if standard C++ had some built-in function that made this easy.
The simplest way to solve your problem is probably to impose some additional constraints, such as requiring that text files have a name ending with ".txt" or that you only guarantee correct operation if no none ASCII files are in the directory.
If you are running on a POSIX system you can also use the 'file' program which tries to figure out what kind of contents a file has.
On Jul 6, 3:52 am, Sam <s...@email-scan.com> wrote:
> Brad writes: > > That lead to my initial question. Later I could learn how to > > deal with binary files that I may want to search like PDF > > and MS Office documents. Just curious if standard C++ had > > some built-in function that made this easy. > No. The only 'built-in' function of any kind is one to test if > a single character belongs in a given character class: > isascii() and its equivalents. It's up to you to scan the > entire contents of the file, to classify it.
There is no isascii function, and the other isxxx functions are locale dependent (and don't really work for narrow characters anyway). There are heuristics for "guessing" the type of contents of a file, but they're just that, heuristics, and none are 100% certain.
Most systems have various conventions which may reveal the type, but those are also just conventions, and individual files may actually violate them: you can give a text file an name ending with .exe under Windows, and there's nothing to prevent a binary file from starting with something that looks like like "<!DOCTYPE..." on any system.
> In POSIX, you might be able to get away with opening a file, > stat()ing its contents, to get the file's size, mmap-ing the > file into memory, then using std::find_if() to search for > non-ascii bytes. Of course, if you hit a 4gb file, that might > cause ...problems.
Under most Unix systems, you'd probably read the first N bytes (maybe 512, although that's a lot more than would typically be necessary), and then exploit magic. For that matter, *generally*, reading the first 512 bytes, then looking for characters outside the set 0x07-0x0D and 0x20-0x7E, is probably a pretty good heuristic; the probability of your guessing wrong is pretty slim (but of course, it will treat non-ascii text files as binary).
-- James Kanze (GABI Software) email:james.ka...@gmail.com Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
