strtok在C中的作用是什么?(复制)

This question already has an answer here:

这个问题已经有了答案:

How does strtok() split the string into tokens in C? 13 answers
strtok()如何将字符串分割为C中的记号?13个答案

I found this sample program which explains the strtok function:

我找到了这个解释strtok函数的示例程序:

#include <stdio.h>
#include <string.h>

int main ()
{
    char str[] ="- This, a sample string.";
    char * pch;
    printf ("Splitting string \"%s\" into tokens:\n",str);
    pch = strtok (str," ,.-");
    while (pch != NULL)
    {
        printf ("%s\n",pch);
        pch = strtok (NULL, " ,.-");
    }
    return 0;
}

However, I don't see how this is possible to work.

然而，我不认为这是可能的。

How is it possible that pch = strtok (NULL, " ,.-"); returns a new token. I mean, we are calling strtokwith NULL . This doesen't make a lot sense to me.

pch = strtok (NULL，“。-”)的可能性有多大?返回一个新的令牌。我们调用strtokwithnull。这对我来说没什么意义。

4 个解决方案

#1

Two things to know about strtok. As was mentioned, it "maintains internal state". Also, it messes up the string you feed it. Essentially, it will write a '\0' where it finds the token you supplied, and returns a pointer to the start of the string. Internally it maintains the location of the last token; and next time you call it, it starts from there.

关于strtok有两件事需要了解。如前所述，它“保持内部状态”。而且，它会打乱你输入的字符串。本质上，它将在找到您提供的令牌的地方写入'\0'，并返回指向字符串开始的指针。在内部，它维护最后一个令牌的位置;下次你调用它时，它从这里开始。

The important corollary is that you cannot use strtok on a const char* "hello world"; type of string, since you will get an access violation when you modify contents of a const char* string.

重要的推论是你不能在const char*“hello world”中使用strtok;字符串的类型，因为当您修改const char* string的内容时，会遇到访问冲突。

The "good" thing about strtok is that it doesn't actually copy strings - so you don't need to manage additional memory allocation etc. But unless you understand the above, you will have trouble using it correctly.

strtok的“好处”是它实际上不复制字符串——所以您不需要管理额外的内存分配等等。但是除非您理解上面的内容，否则您将无法正确地使用它。

Example - if you have "this,is,a,string", successive calls to strtok will generate pointers as follows (the ^ is the value returned). Note that the '\0' is added where the tokens are found; this means the source string is modified:

例子,如果你有,一个是字符串,连续调用strtok指针将生成如下(^是返回值)。注意，在找到令牌的地方添加了'\0';这意味着源字符串被修改:

t  h  i  s  ,  i  s  ,  a  ,  s  t  r  i  n  g \0         this,is,a,string

t  h  i  s  \0 i  s  ,  a  ,  s  t  r  i  n  g \0         this
^
t  h  i  s  \0 i  s  \0 a  ,  s  t  r  i  n  g \0         is
               ^
t  h  i  s  \0 i  s  \0 a  \0 s  t  r  i  n  g \0         a
                        ^
t  h  i  s  \0 i  s  \0 a  \0 s  t  r  i  n  g \0         string
                              ^

Hope it makes sense.

希望这是有意义的。

#2

strtok maintains internal state. When you call it with non-NULL it re-initializes itself to use the string you supply. When you call it with NULL it uses that string, and any other state its currently got to return the next token.

strtok维护内部状态。当您调用非null时，它会重新初始化自己以使用您提供的字符串。当你用NULL调用它时，它会使用这个字符串，并且它当前的任何状态都要返回下一个令牌。

Because of the way strtok works you need to ensure that you link with a multithreaded version of the C runtime if you're writing a multithreaded application. This will ensure that each thread get its own internal state for strtok.

由于strtok的工作方式，您需要确保在编写多线程应用程序时与C运行时的多线程版本进行链接。这将确保每个线程都有自己的strtok内部状态。

#3

The strtok() function stores data between calls. It uses that data when you call it with a NULL pointer.

函数的作用是:在调用之间存储数据。当你用一个空指针调用它时，它会使用那个数据。

From http://www.cplusplus.com/reference/cstring/strtok/ :

从http://www.cplusplus.com/reference/cstring/strtok/:

The point where the last token was found is kept internally by the function to be used on the next call (particular library implementations are not required to avoid data races).

找到最后一个令牌的点是由将在下一个调用中使用的函数在内部保留的(不需要特定的库实现来避免数据竞争)。

#4

The strtok function stores data in an internal static variable which is shared among all threads.

strtok函数将数据存储在一个内部静态变量中，该变量在所有线程之间共享。

For thread safety you should use strtok_r

为了线程安全，您应该使用strtok_r

From http://www.opensource.apple.com/source/Libc/Libc-167/string.subproj/strtok.c

从http://www.opensource.apple.com/source/Libc/Libc-167/string.subproj/strtok.c

Take a look to static char *last;

最后看看静态char *;

char *
strtok(s, delim)
    register char *s;
    register const char *delim;
{
    register char *spanp;
    register int c, sc;
    char *tok;
    static char *last;


    if (s == NULL && (s = last) == NULL)
        return (NULL);

    /*
     * Skip (span) leading delimiters (s += strspn(s, delim), sort of).
     */
cont:
    c = *s++;
    for (spanp = (char *)delim; (sc = *spanp++) != 0;) {
        if (c == sc)
            goto cont;
    }

    if (c == 0) {       /* no non-delimiter characters */
        last = NULL;
        return (NULL);
    }
    tok = s - 1;

    /*
     * Scan token (scan for delimiters: s += strcspn(s, delim), sort of).
     * Note that delim must have one NUL; we stop if we see that, too.
     */
    for (;;) {
        c = *s++;
        spanp = (char *)delim;
        do {
            if ((sc = *spanp++) == c) {
                if (c == 0)
                    s = NULL;
                else
                    s[-1] = 0;
                last = s;
                return (tok);
            }
        } while (sc != 0);
    }
    /* NOTREACHED */
}

#1