How to Read in Chunks of Text C++
Solarian Programmer
My programming ramblings
C Programming - read a file line past line with fgets and getline, implement a portable getline version
Posted on April three, 2019 by Paul
In this article, I will show you how to read a text file line by line in C using the standard C part fgets and the POSIX getline function. At the finish of the article, I will write a portable implementation of the getline part that can be used with whatever standard C compiler.
Reading a file line by line is a trivial trouble in many programming languages, but non in C. The standard manner of reading a line of text in C is to utilize the fgets part, which is fine if you know in accelerate how long a line of text could be.
You tin can find all the code examples and the input file at the GitHub repo for this commodity.
Permit's start with a simple case of using fgets to read chunks from a text file. :
ane #include <stdio.h> 2 #include <stdlib.h> 3 4 int main ( void ) { v FILE * fp = fopen ( "lorem.txt" , "r" ); vi if ( fp == Cipher ) { 7 perror ( "Unable to open file!" ); eight go out ( 1 ); ix } x 11 char chunk [ 128 ]; 12 xiii while ( fgets ( chunk , sizeof ( chunk ), fp ) != NULL ) { xiv fputs ( clamper , stdout ); 15 fputs ( "|* \northward " , stdout ); // marker string used to show where the content of the chunk array has ended 16 } 17 18 fclose ( fp ); 19 } For testing the code I've used a simple dummy file, lorem.txt. This is a piece from the output of the to a higher place program on my car:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t0.c -o t0 2 ~ $ ./t0 3 Lorem ipsum dolor sit amet, consectetur adipiscing elit. 4 |* 5 Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare cond|* half-dozen imentum. 7 |* viii Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien |* 9 dignissim molestie. 10 |* The lawmaking prints the content of the chunk assortment, equally filled subsequently every call to fgets, and a marking cord.
If you watch advisedly, by scrolling the in a higher place text snippet to the right, y'all can see that the output was truncated to 127 characters per line of text. This was expected because our code can store an entire line from the original text file merely if the line can fit within our clamper array.
What if you demand to have the unabridged line of text bachelor for further processing and not a piece of line ? A possible solution is to copy or concatenate chunks of text in a separate line buffer until we find the end of line grapheme.
Permit's start by creating a line buffer that will store the chunks of text, initially this volition have the aforementioned length as the clamper array:
i #include <stdio.h> 2 #include <stdlib.h> 3 #include <string.h> 4 5 int master ( void ) { 6 FILE * fp = fopen ( "lorem.txt" , "r" ); 7 // ... 8 9 char clamper [ 128 ]; 10 11 // Store the chunks of text into a line buffer 12 size_t len = sizeof ( chunk ); 13 char * line = malloc ( len ); 14 if ( line == NULL ) { 15 perror ( "Unable to allocate memory for the line buffer." ); 16 exit ( one ); 17 } 18 19 // "Empty" the string xx line [ 0 ] = '\0' ; 21 22 // ... 23 24 } Next, we are going to append the content of the chunk array to the end of the line string, until we find the end of line character. If necessary, we'll resize the line buffer:
one #include <stdio.h> 2 #include <stdlib.h> 3 #include <string.h> iv 5 int main ( void ) { 6 // ... seven 8 // "Empty" the cord 9 line [ 0 ] = '\0' ; x 11 while ( fgets ( chunk , sizeof ( chunk ), fp ) != NULL ) { 12 // Resize the line buffer if necessary thirteen size_t len_used = strlen ( line ); 14 size_t chunk_used = strlen ( chunk ); 15 sixteen if ( len - len_used < chunk_used ) { 17 len *= 2 ; 18 if (( line = realloc ( line , len )) == NULL ) { 19 perror ( "Unable to reallocate memory for the line buffer." ); 20 free ( line ); 21 exit ( 1 ); 22 } 23 } 24 25 // Copy the chunk to the end of the line buffer 26 strncpy ( line + len_used , chunk , len - len_used ); 27 len_used += chunk_used ; 28 29 // Check if line contains '\n', if yes process the line of text xxx if ( line [ len_used - 1 ] == '\n' ) { 31 fputs ( line , stdout ); 32 fputs ( "|* \n " , stdout ); 33 // "Empty" the line buffer 34 line [ 0 ] = '\0' ; 35 } 36 } 37 38 fclose ( fp ); 39 gratuitous ( line ); 40 41 printf ( " \n\n Max line size: %zd \northward " , len ); 42 } Please note, that in the above lawmaking, every time the line buffer needs to be resized its capacity is doubled.
This is the event of running the to a higher place code on my machine. For brevity, I kept only the get-go lines of output:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 two ~ $ ./t1 iii Lorem ipsum dolor sit amet, consectetur adipiscing elit. 4 |* 5 Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare condimentum. 6 |* 7 Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien dignissim molestie. viii |* 9 Aliquam erat volutpat. Mauris dignissim augue ac purus placerat scelerisque. Donec eleifend ut nibh eu elementum. 10 |* You can meet that, this time, we can print full lines of text and not fixed length chunks like in the initial approach.
Let'south alter the above code in order to print the line length instead of the actual text:
i // ... 2 3 int chief ( void ) { 4 // ... 5 six while ( fgets ( chunk , sizeof ( chunk ), fp ) != Zero ) { seven 8 // ... ix ten // Check if line contains '\n', if yes process the line of text 11 if ( line [ len_used - 1 ] == '\n' ) { 12 printf ( "line length: %zd \n " , len_used ); 13 // "Empty" the line buffer 14 line [ 0 ] = '\0' ; 15 } 16 } 17 18 fclose ( fp ); 19 costless ( line ); 20 21 printf ( " \n\n Max line size: %zd \north " , len ); 22 } This is the result of running the modified code on my car:
i ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 2 ~ $ ./t1 3 line length: 57 4 line length: 136 v line length: 147 half dozen line length: 114 7 line length: 112 8 line length: 95 9 line length: 62 ten line length: 1 11 line length: 428 12 line length: 1 13 line length: 460 fourteen line length: 1 15 line length: 834 16 line length: 1 17 line length: 821 18 nineteen 20 Max line size: 1024 In the adjacent example, I will show you lot how to use the getline office bachelor on POSIX systems like Linux, Unix and macOS. Microsoft Visual Studio doesn't have an equivalent function, then you lot won't be able to easily examination this example on a Windows organisation. Even so, you should be able to test information technology if you are using Cygwin or Windows Subsystem for Linux.
1 #include <stdio.h> two #include <stdlib.h> iii #include <cord.h> iv 5 int chief ( void ) { half-dozen FILE * fp = fopen ( "lorem.txt" , "r" ); 7 if ( fp == Nil ) { 8 perror ( "Unable to open up file!" ); 9 go out ( 1 ); 10 } 11 12 // Read lines using POSIX function getline 13 // This code won't work on Windows 14 char * line = Cipher ; 15 size_t len = 0 ; 16 17 while ( getline ( & line , & len , fp ) != - i ) { 18 printf ( "line length: %zd \north " , strlen ( line )); 19 } twenty 21 printf ( " \north\n Max line size: %zd \n " , len ); 22 23 fclose ( fp ); 24 free ( line ); // getline will resize the input buffer as necessary 25 // the user needs to free the memory when non needed! 26 } Delight notation, how elementary is to use POSIX'due south getline versus manually buffering chunks of line similar in my previous instance. It is unfortunate that the standard C library doesn't include an equivalent role.
When you use getline, don't forget to gratuitous the line buffer when you don't need it anymore. Also, calling getline more than in one case will overwrite the line buffer, brand a re-create of the line content if you need to keep it for further processing.
This is the result of running the in a higher place getline instance on a Linux machine:
1 ~ $ clang -std=gnu17 -Wall -Wextra -pedantic t2.c -o t2 2 ~ $ ./t2 3 line length: 57 4 line length: 136 5 line length: 147 half-dozen line length: 114 7 line length: 112 8 line length: 95 9 line length: 62 10 line length: one 11 line length: 428 12 line length: i 13 line length: 460 fourteen line length: 1 15 line length: 834 16 line length: one 17 line length: 821 18 19 20 Max line size: 960 Information technology is interesting to notation, that for this particular instance the getline function on Linux resizes the line buffer to a max of 960 bytes. If y'all run the aforementioned code on macOS the line buffer is resized to 1024 bytes. This is due to the different ways in which getline is implemented on different Unix similar systems.
As mentioned before, getline is non present in the C standard library. It could be an interesting do to implement a portable version of this function. The idea here is non to implement the most performant version of getline, but rather to implement a unproblematic replacement for non POSIX systems.
We are going to take the in a higher place case and supercede the POSIX'southward getline version with our own implementation, say my_getline. Obviously, if you lot are on a POSIX system, you should use the version provided by the operating system, which was tested by countless users and tuned for optimal performance.
The POSIX getline function has this signature:
1 ssize_t getline ( char ** restrict lineptr , size_t * restrict n , FILE * restrict stream ); Since ssize_t is too a POSIX defined type, ordinarily a 64 bits signed integer, this is how we are going to declare our version:
1 int64_t my_getline ( char ** restrict line , size_t * restrict len , FILE * restrict fp ); In principle we are going to implement the function using the same arroyo as in 1 of the above examples, where I've defined a line buffer and kept copying chunks of text in the buffer until we found the end of line character:
i // This will only have effect on Windows with MSVC 2 #ifdef _MSC_VER 3 #ascertain _CRT_SECURE_NO_WARNINGS 1 4 #ascertain restrict __restrict 5
0 Response to "How to Read in Chunks of Text C++"
Post a Comment