Protecting Executable Files From Compromising Security

Programs read and write data on the stack. Typically, they execute from read-only portions of memory that are specifically designated for code. Some attacks that cause buffers on the stack to overflow try to insert new code on the stack and cause the program to execute it. Removing execute permission from the stack memory prevents these attacks from succeeding. That is, most programs can function correctly without using executable stacks.

64-bit processes always have non-executable stacks. By default, 32-bit SPARC processes have executable stacks. The noexec_user_stack variable enables you to specify whether the stacks of 32-bit processes are executable.

Once this variable is set, programs that attempt to execute code on their stack are sent a SIGSEGV signal. This signal usually results in the program terminating with a core dump. Such programs also generate a warning message that includes the name of the offending program, the process ID, and the real UID of the user who ran the program. For example:

a.out[347] attempt to execute code on stack by uid 555 

The message is logged by the syslog daemon when the syslog kern facility is set to notice level. This logging is set by default in the syslog.conf file, which means that the message is sent to both the console and the /var/adm/messages file. For more information, see the syslogd(1M) and syslog.conf(4) man pages.

The syslog message is useful for observing potential security problems. The message also identifies valid programs that depend upon executable stacks that have been prevented from correct operation by setting the noexec_user_stack variable. If you do not want any messages logged, then set the log variable, noexec_user_stack_log, to zero in the /etc/system file. Even though messages are not being logged, the SIGSEGV signal can continue to cause the executing program to terminate with a core dump.

Programs can explicitly mark or prevent stack execution. The mprotect() function in programs explicitly marks the stack as executable. For more information, see the mprotect(2) man page. A program compiled with -M /usr/lib/ld/map.noexstk makes the stack non-executable regardless of the system-wide setting.