为什么会报错 malloc: invalid size (unsorted)?

时间:2024-10-14 11:06:48

问题:为什么会报错 malloc(): invalid size (unsorted)?

我在/JamesRead5737/webcrawler/blob/master/有一个网络爬虫代码,它产生了一些我无法解释的奇怪错误。最常见的是它中止并出现错误malloc(): invalid size (unsorted)

回溯显示:

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/:50
#1  0x00007ffff760e859 in __GI_abort () at :79
#2  0x00007ffff76793ee in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff77a3285 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#3  0x00007ffff768147c in malloc_printerr (str=str@entry=0x7ffff77a5a50 "malloc(): invalid size (unsorted)") at :5347
#4  0x00007ffff7684234 in _int_malloc (av=av@entry=0x7ffff77d4b80 <main_arena>, bytes=bytes@entry=8200) at :3736
#5  0x00007ffff7686419 in __GI___libc_malloc (bytes=8200) at :3066
#6  0x00005555555578b3 in html_link_find (url=0x55555d0f8b08 "/intl/en_us//security-center/", 
    html=0x55555f9e6c00 "<!DOCTYPE html>\n<html lang=\"en\" dir=\"ltr\">\n  <head>\n    <meta charset=\"utf-8\">\n    <title>Android Security Center</title>\n    <meta content=\"initial-scale=1, minimum-scale=1, width=device-width\" name="...) at :455
#7  0x0000555555557d70 in html_parse (url=0x55555d0f8b08 "/intl/en_us//security-center/", 
    html=0x55555f9e6c00 "<!DOCTYPE html>\n<html lang=\"en\" dir=\"ltr\">\n  <head>\n    <meta charset=\"utf-8\">\n    <title>Android Security Center</title>\n    <meta content=\"initial-scale=1, minimum-scale=1, width=device-width\" name="...) at :536
#8  0x00005555555582cc in check_multi_info (g=0x7ffffffe0970) at :678
#9  0x00005555555583db in event_cb (g=0x7ffffffe0970, fd=1164, revents=1) at :706
#10 0x0000555555559829 in crawler_init () at :1154
#11 0x0000555555559ae9 in main (argc=1, argv=0x7fffffffe018) at :1207

这将我带到写着sql_current->next = (SqlNode *)malloc(sizeof(SqlNode));的代码行,据我所知,这应该不会导致任何错误。

谷歌搜索错误表明问题可能出现在代码中的任何位置,并且与所讨论的行完全无关。那正确吗?

MySQL数据库设置如下:

USE crawl;
CREATE TABLE IF NOT EXISTS `crawled` (`id` int NOT NULL AUTO_INCREMENT, `url` varchar(768) DEFAULT NULL, `title` varchar(768) DEFAULT NULL, `date` varchar(128) DEFAULT NULL, `links` int DEFAULT 0, `backlinks` int DEFAULT 0, `frontier` int DEFAULT 1, PRIMARY KEY (`id`), UNIQUE KEY `url` (`url`), KEY `title` (`title`), KEY `frontier` (`frontier`)) ENGINE=InnoDB AUTO_INCREMENT=1;
CREATE TABLE IF NOT EXISTS `emails` (`email` varchar(2084) NOT NULL, `id` int NOT NULL AUTO_INCREMENT, PRIMARY KEY (`id`), UNIQUE KEY `email` (`email`)) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
INSERT INTO crawled (url) VALUES (''),(''),('');

如果有问题的线路很好,我怎么能找到真正的问题?

编辑

这是一些 valgrind 输出:

==318618== Memcheck, a memory error detector
==318618== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==318618== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==318618== Command: ./
==318618== Parent PID: 2591
==318618== 
==318618== Warning: ignored attempt to set SIGKILL handler in sigaction();
==318618==          the SIGKILL signal is uncatchable
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10D052: crawler_init (:987)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe28ac is on thread 1's stack
==318618==  in frame #1, created by crawler_init (:956)
==318618== 
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10C61C: setsock (:769)
==318618==    by 0x10C6BC: addsock (:782)
==318618==    by 0x10CE70: sock_cb (:921)
==318618==    by 0x48B70B1: singlesocket (:2593)
==318618==    by 0x48B7878: multi_socket (:2839)
==318618==    by 0x48B8053: curl_multi_socket_action (:2956)
==318618==    by 0x10C4E8: timer_cb (:741)
==318618==    by 0x10D7C7: crawler_init (:1152)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe2584 is on thread 1's stack
==318618==  in frame #1, created by setsock (:749)
==318618== 
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10C61C: setsock (:769)
==318618==    by 0x10CE8F: sock_cb (:923)
==318618==    by 0x48B70B1: singlesocket (:2593)
==318618==    by 0x48B7878: multi_socket (:2839)
==318618==    by 0x48B8053: curl_multi_socket_action (:2956)
==318618==    by 0x10C3BA: event_cb (:703)
==318618==    by 0x10D828: crawler_init (:1154)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe25a4 is on thread 1's stack
==318618==  in frame #1, created by setsock (:749)
==318618== 
==318618== 
==318618== HEAP SUMMARY:
==318618==     in use at exit: 149,695,831 bytes in 27,400 blocks
==318618==   total heap usage: 2,198,504 allocs, 2,171,104 frees, 3,507,931,785 bytes allocated
==318618== 
==318618== LEAK SUMMARY:
==318618==    definitely lost: 1,889,627 bytes in 9,067 blocks
==318618==    indirectly lost: 0 bytes in 0 blocks
==318618==      possibly lost: 2,137,116 bytes in 27 blocks
==318618==    still reachable: 145,669,088 bytes in 18,306 blocks
==318618==         suppressed: 0 bytes in 0 blocks
==318618== Rerun with --leak-check=full to see details of leaked memory
==318618== 
==318618== Use --track-origins=yes to see where uninitialised values come from
==318618== ERROR SUMMARY: 295 errors from 3 contexts (suppressed: 0 from 0)
==318618== 
==318618== 1 errors in context 1 of 3:
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10D052: crawler_init (:987)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe28ac is on thread 1's stack
==318618==  in frame #1, created by crawler_init (:956)
==318618== 
==318618== 
==318618== 79 errors in context 2 of 3:
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10C61C: setsock (:769)
==318618==    by 0x10CE8F: sock_cb (:923)
==318618==    by 0x48B70B1: singlesocket (:2593)
==318618==    by 0x48B7878: multi_socket (:2839)
==318618==    by 0x48B8053: curl_multi_socket_action (:2956)
==318618==    by 0x10C3BA: event_cb (:703)
==318618==    by 0x10D828: crawler_init (:1154)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe25a4 is on thread 1's stack
==318618==  in frame #1, created by setsock (:749)
==318618== 
==318618== 
==318618== 215 errors in context 3 of 3:
==318618== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==318618==    at 0x515AACE: epoll_ctl (:78)
==318618==    by 0x10C61C: setsock (:769)
==318618==    by 0x10C6BC: addsock (:782)
==318618==    by 0x10CE70: sock_cb (:921)
==318618==    by 0x48B70B1: singlesocket (:2593)
==318618==    by 0x48B7878: multi_socket (:2839)
==318618==    by 0x48B8053: curl_multi_socket_action (:2956)
==318618==    by 0x10C4E8: timer_cb (:741)
==318618==    by 0x10D7C7: crawler_init (:1152)
==318618==    by 0x10DAE8: main (:1207)
==318618==  Address 0x1ffefe2584 is on thread 1's stack
==318618==  in frame #1, created by setsock (:749)
==318618== 
==318618== ERROR SUMMARY: 295 errors from 3 contexts (suppressed: 0 from 0)

编辑

这是崩溃的一些 valgrind 输出:

==319842== Memcheck, a memory error detector
==319842== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==319842== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==319842== Command: ./
==319842== Parent PID: 2591
==319842== 
==319842== Warning: ignored attempt to set SIGKILL handler in sigaction();
==319842==          the SIGKILL signal is uncatchable
==319842== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==319842==    at 0x515AACE: epoll_ctl (:78)
==319842==    by 0x10D052: crawler_init (:987)
==319842==    by 0x10DAE8: main (:1207)
==319842==  Address 0x1ffefe28ac is on thread 1's stack
==319842==  in frame #1, created by crawler_init (:956)
==319842== 
==319842== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==319842==    at 0x515AACE: epoll_ctl (:78)
==319842==    by 0x10C61C: setsock (:769)
==319842==    by 0x10C6BC: addsock (:782)
==319842==    by 0x10CE70: sock_cb (:921)
==319842==    by 0x48B70B1: singlesocket (:2593)
==319842==    by 0x48B7878: multi_socket (:2839)
==319842==    by 0x48B8053: curl_multi_socket_action (:2956)
==319842==    by 0x10C4E8: timer_cb (:741)
==319842==    by 0x10D7C7: crawler_init (:1152)
==319842==    by 0x10DAE8: main (:1207)
==319842==  Address 0x1ffefe2584 is on thread 1's stack
==319842==  in frame #1, created by setsock (:749)
==319842== 
==319842== Syscall param epoll_ctl(event) points to uninitialised byte(s)
==319842==    at 0x515AACE: epoll_ctl (:78)
==319842==    by 0x10C61C: setsock (:769)
==319842==    by 0x10CE8F: sock_cb (:923)
==319842==    by 0x48B70B1: singlesocket (:2593)
==319842==    by 0x48B7878: multi_socket (:2839)
==319842==    by 0x48B8053: curl_multi_socket_action (:2956)
==319842==    by 0x10C3BA: event_cb (:703)
==319842==    by 0x10D828: crawler_init (:1154)
==319842==    by 0x10DAE8: main (:1207)
==319842==  Address 0x1ffefe25a4 is on thread 1's stack
==319842==  in frame #1, created by setsock (:749)
==319842== 
==319842== Invalid write of size 1
==319842==    at 0x48436E4: mempcpy (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck)
==319842==    by 0x50CD1D8: _IO_default_xsputn (:386)
==319842==    by 0x50CD1D8: _IO_default_xsputn (:370)
==319842==    by 0x50B227B: __vfprintf_internal (:1688)
==319842==    by 0x50C0278: __vsprintf_internal (:95)
==319842==    by 0x509D047: sprintf (:30)
==319842==    by 0x10B88F: html_link_find (:452)
==319842==    by 0x10BD6F: html_parse (:536)
==319842==    by 0x10C2CB: check_multi_info (:678)
==319842==    by 0x10C3DA: event_cb (:706)
==319842==    by 0x10D828: crawler_init (:1154)
==319842==    by 0x10DAE8: main (:1207)
==319842==  Address 0xf107d18 is 0 bytes after a block of size 8,200 alloc'd
==319842==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck)
==319842==    by 0x10B736: html_link_find (:440)
==319842==    by 0x10BD6F: html_parse (:536)
==319842==    by 0x10C2CB: check_multi_info (:678)
==319842==    by 0x10C3DA: event_cb (:706)
==319842==    by 0x10D828: crawler_init (:1154)
==319842==    by 0x10DAE8: main (:1207)
==319842== 

valgrind: m_mallocfree.c:305 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 8272, hi = 3625731377157460067.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==319842==    at 0x58046FFA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x58047127: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x580472CB: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x580514B4: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x5803DE9A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x5803CD9F: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x58041F04: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x5803C1D8: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==319842==    by 0x1002EC6B18: ???
==319842==    by 0x1002CADF2F: ???
==319842==    by 0x1002CADF17: ???
==319842==    by 0x1002CADF2F: ???
==319842==    by 0x1002CADF3F: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 319842)
==319842==    at 0x50CD2B4: _IO_default_xsputn (:394)
==319842==    by 0x50CD2B4: _IO_default_xsputn (:370)
==319842==    by 0x50B2165: __vfprintf_internal (:1719)
==319842==    by 0x50C0278: __vsprintf_internal (:95)
==319842==    by 0x509D047: sprintf (:30)
==319842==    by 0x10B88F: html_link_find (:452)
==319842==    by 0x10BD6F: html_parse (:536)
==319842==    by 0x10C2CB: check_multi_info (:678)
==319842==    by 0x10C3DA: event_cb (:706)
==319842==    by 0x10D828: crawler_init (:1154)
==319842==    by 0x10DAE8: main (:1207)
client stack range: [0x1FFEFDB000 0x1FFF000FFF] client SP: 0x1FFEFDB600
valgrind stack range: [0x1002BAE000 0x1002CADFFF] top usage: 10344 of 1048576


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: 

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

解答

将评论转移到答案。

似乎函数html_link_find()在第 440 行分配了内存,然后在第 452 行调用了sprintf()来格式化某些东西,并且它将分配的内存的末尾覆盖了 1 个字节,这足以杀死malloc()

一种部分解决方法是使用snprintf()而不是sprintf()— 但您还需要测试返回值以避免数据截断。有可能某些应该以空值终止的数据不是,这可能导致部分溢出。