docker中部署Python程序--pandas读取excel出现编码错误

2019年8月1日 20:03 by scott

python调试

问题出现的过程

(1)编写Python程序:使用pandas读取excel数据,分析处理后把结果以邮件形式发送。

(2)编写Dockerfile文件,在Docker中部署程序

(3)在Docker中启动定时任务,每天的9点发送邮件

(4)通过定时任务跑出程序时,老是报错:

File "t.py", line 8, in <module>
    df = pd.read_excel(fn)
  File "/usr/local/lib/python3.6/site-packages/pandas/util/_decorators.py", line 178, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pandas/util/_decorators.py", line 178, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pandas/io/excel.py", line 307, in read_excel
    io = ExcelFile(io, engine=engine)
  File "/usr/local/lib/python3.6/site-packages/pandas/io/excel.py", line 394, in __init__
    self.book = xlrd.open_workbook(self._io)
  File "/usr/local/lib/python3.6/site-packages/xlrd/__init__.py", line 116, in open_workbook
    with open(filename, "rb") as f:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 10-16: ordinal not in range(128)

猜测原因:sheet name出现了中文,或者sheet 内容中出现了中文,事实上都出现了。

但是进入Docker中手动运行是没有问题的。原因出现在了哪儿呢?

肯定是定时任务运行时的环境和手动执行时环境不一致导致的

尝试各种解决办法

可是究竟该怎么把环境配一致呢,尝试了以下方法:

(1)由于是编码导致的,那么就解决编码问题,在定时任务中加上编码方式(PYTHONIOENCODING=utf-8),如下(未凑效):

1 09 * * * cd /code && PYTHONIOENCODING=utf-8 /usr/local/bin/python send_email.py >> /var/log/cron.log 2>&1

(2)既然是shell环境导致,那么把运行命令放入shell脚本中运行(同时把脚本配到crontab中,此处略),shell脚本内容如下(为凑效):

#!/bin/sh
# 为了使python版本一致,这里用的是python3
. /etc/profile
echo "good"
echo `python --version`
python send_email.py 

最终解决方案

 这里先列出解决方案(最终的shell脚本):

#!/bin/sh
. /etc/profile
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
echo "good"
echo `python --version`
python send_email.py

感兴趣的可以继续往下看解决问题的思路。

(1)既然是编码导致的问题,手动可运行,定时任务中不行,那么我把编码打出来看看是啥。

在Python脚本中加入以下行来打印编码内容(系统默认编码、文件转换编码):

import sys
print("system coding:",sys.getdefaultencoding())
print("file coding:",sys.getfilesystemencoding())

(2)再次手动执行,再次定时任务中执行,输出分别如下:

# 手动运行时的输出
system coding: utf-8
file coding: utf-8

# 定时任务中的输出
system coding: utf-8
file coding: ascii

(3) 为什么不一样呢,最后到网上查询,决定 sys.getfilesystemencoding() 值的shell变量是 LANGLC_* ,  原文描述如下:

Terminals usually spawns a new shell and setting LANG or LC_* environment variables will affect only programs that will run withing this shell.

于是在shell脚本中添加如下行:

export LC_ALL=C.UTF-8
export LANG=C.UTF-8

 (4)再次在定时任务中执行shell脚本,最终成功运行!激动。。。成就感。。。。

总结

(1)要善于从案发一线找线索,就像警察破案:光知道大致方向,找不到线索也是白搭。

(2)关于Linux的知识,知道的越多越好。比如这里的shell环境中的编码怎么设置,以及有哪些影响。

(3)python中有些工具包、工具函数能帮助我们查找问题。

(4)善于使用Google(百度)搜索引擎。

后续

(1)如有其他问题,欢迎留言。

(2)如果想知道怎么在Docker中配置定时任务,欢迎留言。

(3)想知道怎么发邮件、怎么用Docker部署程序,欢迎留言。

ps. 也许这个问题对你来说不算什么,能把解决过程整理出来,是不是也值得鼓励呢?


Comments(127) Add Your Comment

Charlesnag
“I haven’t seen you in these parts,” the barkeep said, sidling over to where I sat. “Designation’s Bao.” He stated it exuberantly, as if solemn word of honour of his exploits were shared aside settlers hither multifarious a ‚lan in Aeternum. He waved to a wooden keg apart from us, and I returned his gesticulate with a nod. He filled a field-glasses and slid it to me across the stained red wood of the court before continuing. “As a betting chains, I’d be assenting to wager a adequate speck of silver you’re in Ebonscale Reach for the purpose more than the drink and sights,” he said, eyes glancing from the sword sheathed on my cool to the bend slung across my back. http://www.google.gy/url?q=https://renewworld.ru/
Ewuufk
brand atorvastatin 20mg <a href="https://lipiws.top/">order atorvastatin 10mg generic</a> buy lipitor 20mg pills
Iwoolg
order ciprofloxacin 1000mg - <a href="https://cipropro.top/">purchase cipro generic</a> buy clavulanate generic
Fxmbcu
baycip cost - <a href="https://metroagyl.top/keflexcapsule/">cephalexin usa</a> order generic augmentin 625mg
Wijgbl
buy flagyl without a prescription - <a href="https://metroagyl.top/azithromycin/">azithromycin cost</a> zithromax 250mg ca
Rnbfhr
buy ciplox 500mg generic - <a href="https://septrim.top/trimox/">buy amoxicillin online</a> erythromycin 250mg tablet
Fgoaxr
valacyclovir for sale - <a href="https://gnantiviralp.com/nemasole/">buy nemasole online</a> order acyclovir 400mg online
Pykwrv
stromectol online pharmacy - <a href="https://keflexin.top/ceftin/">buy axetil for sale</a> tetracycline brand
Kiyajg
order metronidazole 200mg pill - <a href="https://metroagyl.top/cefaclor/">buy cefaclor medication</a> buy azithromycin no prescription
Hjouls
buy ampicillin generic <a href="https://ampiacil.top/">how to buy ampicillin</a> buy amoxicillin
Sluyov
buy lasix without prescription - <a href="https://antipathogenc.com/capoten/">captopril online buy</a> capoten 25 mg cost
Saedsz
glycomet 500mg sale - <a href="https://gnantiacidity.com/">glycomet 500mg sale</a> order lincocin 500 mg for sale
Qyiiiy
zidovudine 300mg pill - <a href="https://canadiangnp.com/rulide/">roxithromycin 150 mg price</a> zyloprim brand
Unuatc
clozaril 50mg without prescription - <a href="https://genonlinep.com/amaryl/">cost amaryl</a> purchase famotidine online cheap
Idymfe
buy generic quetiapine - <a href="https://gnkantdepres.com/effexor/">purchase effexor online</a> buy eskalith cheap
Axsxnk
buy clomipramine tablets - <a href="https://antdeponline.com/asendin/">buy generic amoxapine</a> buy generic doxepin over the counter
Yffsqj
buy atarax 10mg online - <a href="https://antdepls.com/endep/">order endep generic</a> amitriptyline oral
Qvjvaj
buy clavulanate pills for sale - <a href="https://atbioinfo.com/linezolid/">buy linezolid 600 mg for sale</a> buy ciprofloxacin 500mg for sale
Clyums
amoxicillin price - <a href="https://atbioxotc.com/cefadroxil/">cefadroxil drug</a> buy baycip cheap
Pwkgmg
cleocin medication - <a href="https://cadbiot.com/">buy cleocin generic</a> chloramphenicol medication
Epzxdp
buy zithromax online cheap - <a href="https://gncatbp.com/btetracycline/">buy generic sumycin online</a> ciplox canada
Fpmakz
ivermectin 3 mg pills for humans - <a href="https://antibpl.com/">buy ivermectin tablets</a> cefaclor order online
Gbxysh
purchase ventolin for sale - <a href="https://antxallergic.com/cvallegra/">generic allegra 180mg</a> how to get theo-24 Cr without a prescription
Gpydyr
buy methylprednisolone canada - <a href="https://ntallegpl.com/getcetirizine/">cetirizine 10mg tablet</a> azelastine 10ml over the counter
Tgager
buy desloratadine 5mg pill - <a href="https://rxtallerg.com/getzaditor/">generic zaditor 1mg</a> buy albuterol inhalator sale
Ilyyjv
glyburide order - <a href="https://prodeprpl.com/dapagliflozin5/">dapagliflozin 10 mg canada</a> forxiga 10 mg brand
Brlswm
glucophage for sale online - <a href="https://arxdepress.com/">buy glycomet 1000mg</a> order acarbose 50mg sale
Blzhpy
buy generic prandin for sale - <a href="https://depressinfo.com/">buy prandin 1mg online</a> buy empagliflozin 10mg for sale
Sdprcw
purchase terbinafine without prescription - <a href="https://treatfungusx.com/fulvicin250mg/">buy griseofulvin</a> griseofulvin without prescription
Clkczp
buy rybelsus 14mg online - <a href="https://infodeppl.com/ddavpspray/">buy desmopressin paypal</a> purchase DDAVP sale
Bykbeg
nizoral 200 mg cheap - <a href="https://antifungusrp.com/">buy nizoral pill</a> generic itraconazole 100 mg
Ydijte
famciclovir 250mg usa - <a href="https://amvinherpes.com/">buy famvir 250mg without prescription</a> order valcivir 500mg sale
Xexhpt
order lanoxin generic - <a href="https://blpressureok.com/dipyridamole100/">cost dipyridamole</a> lasix pills
Nygqpn
hydrochlorothiazide sale - <a href="https://norvapril.com/">microzide 25mg drug</a> buy bisoprolol 10mg for sale
Drgksd
metoprolol 100mg usa - <a href="https://bloodpresspl.com/nifedipine/">generic adalat</a> generic nifedipine 10mg
Vjqfus
nitroglycerin for sale online - <a href="https://nitroproxl.com/indapamide/">lozol pill</a> buy diovan no prescription
Rsvxlg
rosuvastatin rage - <a href="https://antcholesterol.com/">crestor gloomy</a> caduet brass
Mmixhd
zocor happy - <a href="https://canescholest.com/fenofibrate/">fenofibrate rule</a> lipitor chapter
Idtfmw
buy viagra professional job - <a href="https://edsildps.com/malegra/">malegra footstep</a> levitra oral jelly storm
Trbncy
dapoxetine watson - <a href="https://promedprili.com/viagraplus/">viagra plus resume</a> cialis with dapoxetine suspicious
Xyeijo
cenforce stagger - <a href="https://xcenforcem.com/tadaciptadalafil/">buy tadacip 20mg</a> brand viagra online chimney
Mkvkxd
brand cialis panic - <a href="https://probrandtad.com/penisole/">penisole difference</a> penisole wan
Hieojp
cialis soft tabs online yeah - <a href="https://supervalip.com/cialisoraljelly/">cialis oral jelly request</a> viagra oral jelly delight
Qpufir
brand cialis air - <a href="https://probrandtad.com/apcalissx/">apcalis bank</a> penisole dry
Ewbsxx
cialis soft tabs unicorn - <a href="https://supervalip.com/cialissuperactive/">cialis super active online extra</a>1 viagra oral jelly online snatch
Vgrfqg
cenforce clutch - <a href="https://xcenforcem.com/brandviagrasildenafil/">brand viagra convince</a> brand viagra shell
Dqndzn
acne medication swell - <a href="https://placnemedx.com/">acne treatment give</a> acne medication faint
Nlyhtt
asthma medication reflect - <a href="https://bsasthmaps.com/">asthma medication gang</a> inhalers for asthma possible
Kuetgc
uti antibiotics attempt - <a href="https://amenahealthp.com/">uti antibiotics someone</a> uti medication sunday
Ftncha
prostatitis medications potion - <a href="https://xprosttreat.com/">prostatitis pills suspect</a> prostatitis medications hum
Nwjbqq
valtrex blaze - <a href="https://gnantiviralp.com/">valtrex online ghastly</a> valtrex mortal
Gapnwo
claritin pills personal - <a href="https://clatadine.top/">loratadine refresh</a> loratadine value
Ricitj
dapoxetine taste - <a href="https://prilixgn.top/">dapoxetine dirty</a> dapoxetine notion
Nlcwlr
loratadine medication hopeless - <a href="https://clatadine.top/">claritin pills misty</a> claritin pills incredible
Gsewrn
ascorbic acid duck - <a href="https://ascxacid.com/">ascorbic acid demand</a> ascorbic acid list
Nqjgtf
promethazine agent - <a href="https://prohnrg.com/">promethazine sigh</a> promethazine wind
Jgbcoj
biaxin pan - <a href="https://gastropls.com/ranitidineonline/">ranitidine fierce</a> cytotec myself
Iunwpo
fludrocortisone thorough - <a href="https://gastroplusp.com/propantoprazole/">pantoprazole last</a> prevacid lap
Linaat
rabeprazole 20mg over the counter - <a href="https://gastrointesl.com/reglanmetoclop/">metoclopramide 20mg uk</a> buy motilium for sale
Uzhino
buy dulcolax generic - <a href="https://gastroinfop.com/oxybutynin/">buy ditropan 5mg pills</a> liv52 20mg canada
Lfxhsl
order bactrim pills - <a href="https://ketiracet.com/">purchase levetiracetam generic</a> cost tobrex 10mg
Oencrb
where to buy eukroma without a prescription - <a href="https://cerazestrel.com/">buy desogestrel 0.075 mg generic</a> duphaston 10 mg canada
Gpargb
order dapagliflozin 10 mg for sale - <a href="https://precarbos.com/">acarbose pills</a> buy precose medication
Pvyopg
order fulvicin 250mg online cheap - <a href="https://gemfilopi.com/">buy gemfibrozil pills for sale</a> buy lopid 300 mg generic
Mdqksv
dramamine 50mg without prescription - <a href="https://prasilx.com/">buy prasugrel 10mg for sale</a> risedronate 35mg brand
Cgpwso
vasotec sale - <a href="https://doxapisin.com/">buy doxazosin 1mg generic</a> order latanoprost generic
Ygxwul
buy etodolac 600 mg for sale - <a href="https://cilosetal.com/">cilostazol over the counter</a> purchase pletal online cheap
Syzcwo
piroxicam 20mg sale - <a href="https://rivastilons.com/">rivastigmine cost</a> purchase exelon without prescription
Xgbxgl
buy nootropil for sale - <a href="https://nootquin.com/praziquantel/">order praziquantel 600mg generic</a> buy cheap sinemet
Sunudu
hydrea pills - <a href="https://hydroydrinfo.com/methocarbamol/">buy methocarbamol online</a> order robaxin 500mg online cheap
Vgzhlb
divalproex sale - <a href="https://adepamox.com/aspirindipyridamole/">buy generic aggrenox</a> order topamax 200mg sale
Untmsa
disopyramide phosphate sale - <a href="https://anorpica.com/pregabal/">buy pregabalin 150mg pill</a> buy generic chlorpromazine online
Zroppz
cytoxan uk - <a href="https://cycloxalp.com/stavudine/">zerit order</a> order vastarel pill
Cxderm
purchase aldactone for sale - <a href="https://aldantinep.com/phenytoin/">dilantin for sale online</a> naltrexone 50 mg canada
Rlsqnh
order flexeril 15mg sale - <a href="https://abflequine.com/olanzapine/">olanzapine 10mg for sale</a> vasotec medication
Nijdty
buy zofran 4mg generic - <a href="https://azofarininfo.com/procyclidine/">where can i buy kemadrin</a> ropinirole 1mg pills
Oohfrh
buy ascorbic acid 500 mg online cheap - <a href="https://mdacidinfo.com/prochlorperazine/">prochlorperazine order online</a> compro medication
Bnnopa
durex gel where to order - <a href="https://xalaplinfo.com/latanoprosteyedrops/">buy xalatan medication</a> buy generic zovirax
Kxpdjz
buy minoxidil generic - <a href="https://hairlossmedinfo.com/finasteridehl/">buy cheap finasteride</a> propecia 1mg sale
Jpvgpf
leflunomide 10mg uk - <a href="https://infohealthybones.com/">purchase arava online cheap</a> buy cartidin without a prescription
Swujbm
cost atenolol - <a href="https://heartmedinfox.com/carvedilol/">coreg canada</a> buy carvedilol no prescription
Zfpiia
calan 120mg us - <a href="https://infoheartdisea.com/cardizem/">order diltiazem generic</a> tenoretic drug
Vkolxy
purchase atorvastatin generic - <a href="https://infoxheartmed.com/nebivolol/">order nebivolol pill</a> buy bystolic 20mg
Jatpyc
gasex tablets - <a href="https://herbalinfomez.com/diabecon/">where to buy diabecon without a prescription</a> generic diabecon
Ftmoyn
order lasuna pills - <a href="https://infoherbalmz.com/">lasuna sale</a> buy himcolin no prescription
Jqnjbm
cheap norfloxacin pill - <a href="https://gmenshth.com/flutamide/">order eulexin generic</a> confido order online
Omhuoh
order finasteride pill - <a href="https://finmenura.com/doxazosin/">buy cheap generic doxazosin</a> buy uroxatral pills
Tbamcz
speman pills - <a href="https://spmensht.com/">buy speman tablets</a> purchase fincar online
Flsqnw
buy terazosin generic - <a href="https://hymenmax.com/tamsulosin/">cost tamsulosin</a> order priligy 30mg generic
Eikzha
oxcarbazepine us - <a href="https://trileoxine.com/">buy trileptal 300mg</a> brand synthroid 150mcg
Mpennz
how to buy imusporin - <a href="https://asimusxate.com/gloperba/">order colchicine 0.5mg pills</a> buy colchicine pills for sale
Nzsdui
brand lactulose - <a href="https://duphalinfo.com/betahistine/">where can i buy betahistine</a> buy betahistine 16 mg without prescription
Egtbev
calcort online buy - <a href="https://lazacort.com/">order deflazacort pills</a> alphagan order online
Iuxgxu
purchase besivance generic - <a href="https://besifxcist.com/">besivance medication</a> sildamax medication
Ydedgs
gabapentin ca - <a href="https://aneutrin.com/ibuprofen/">motrin 400mg usa</a> azulfidine over the counter
Mgzrhp
buy cheap benemid - <a href="https://bendoltol.com/">brand benemid</a> cheap tegretol
Fqiosk
order generic celebrex 100mg - <a href="https://celespas.com/">celebrex 100mg pills</a> indomethacin 50mg sale
Llvudp
cheap mebeverine 135mg - <a href="https://coloxia.com/etoricoxib/">arcoxia brand</a> buy generic pletal over the counter
Dxkevg
order diclofenac without prescription - <a href="https://dicloltarin.com/">cheap diclofenac</a> brand aspirin 75mg
Xcpqea
purchase rumalaya pill - <a href="https://rumaxtol.com/shallaki/">shallaki tablets</a> order endep 10mg generic
Hclgqc
mestinon online order - <a href="https://mestonsx.com/azathioprine/">azathioprine 50mg without prescription</a> imuran 50mg ca
Sfabzc
order diclofenac pill - <a href="https://vovetosa.com/">generic voveran</a> cheap nimodipine for sale
Vjrdby
order baclofen 10mg sale - <a href="https://baclion.com/nspiroxicam/">purchase feldene sale</a> piroxicam cheap
Kmvori
mobic 15mg canada - <a href="https://meloxiptan.com/rtrizatriptan/">buy toradol online cheap</a> toradol 10mg drug
Upwwqb
artane usa - <a href="https://voltapll.com/diclofenacgel/">purchase diclofenac gel online cheap</a> buy cheap diclofenac gel
Xembow
order omnicef 300mg pill - <a href="https://omnixcin.com/">omnicef uk</a> buy clindamycin sale
Rckvsu
isotretinoin 40mg sale - <a href="https://aisotane.com/asavlosulfon/">order generic dapsone 100mg</a> order deltasone 5mg generic
Rovaov
order deltasone 10mg sale - <a href="https://apreplson.com/">brand prednisone 5mg</a> zovirax oral
Evmjuj
buy permethrin medication - <a href="https://actizacs.com/tretinoin/">order tretinoin gel sale</a> tretinoin order
Xkcqou
buy generic betamethasone 20 gm - <a href="https://betnoson.com/">oral betamethasone 20gm</a> benoquin uk
Eiyoar
flagyl 200mg pills - <a href="https://cenforcevs.com/">cenforce medication</a> order cenforce 100mg online cheap
Ttgiwf
augmentin 1000mg cheap - <a href="https://alevonted.com/">levothyroxine cost</a> order levoxyl online
Aetibb
cleocin oral - <a href="https://clinycinpl.com/">cleocin 300mg ca</a> indocin 50mg cheap
Xmsqaj
oral cozaar 25mg - <a href="https://acephacrsa.com/">keflex price</a> buy keflex tablets
Ouvhhe
how to buy crotamiton - <a href="https://aeuracream.com/caczone/">aczone cheap</a> aczone order
Pyuass
order modafinil generic - <a href="https://sleepagil.com/phenergan/">buy phenergan generic</a> cost melatonin 3mg
Voffdh
bupropion 150mg uk - <a href="https://bupropsl.com/shuddhaguggulu/">shuddha guggulu for sale online</a> where can i buy shuddha guggulu
Nwakmk
order xeloda without prescription - <a href="https://xelocap.com/mefenamicacid/">buy ponstel generic</a> buy danocrine without a prescription
Cwwfhv
prometrium for sale online - <a href="https://apromid.com/">order prometrium 200mg without prescription</a> order clomiphene online
Wzluor
alendronate us - <a href="https://pilaxmax.com/tamoxifen/">where can i buy nolvadex</a> order medroxyprogesterone generic
Pxtihz
order norethindrone sale - <a href="https://norethgep.com/yasmin/">purchase yasmin pills</a> buy cheap generic yasmin
Mklepa
yasmin drug - <a href="https://festrolp.com/">estrace 1mg brand</a> buy arimidex pill
Yexsdk
dostinex 0.25mg over the counter - <a href="https://adostilin.com/">buy cabergoline 0.5mg online cheap</a> cheap alesse pills
Vevxrm
バイアグラ通販おすすめ - <a href="https://jpedpharm.com/tadalafil/">シアリスは薬局で買える?</a> シアリスの購入
Vzvmue
プレドニンの購入 - <a href="https://jpaonlinep.com/">プレドニン通販 安全</a> アジスロマイシン 飲み方
Dizbjk
プレドニンジェネリック 通販 - <a href="https://jpanfarap.com/jpaaccutane/">イソトレチノイン錠 40 mg 強さ</a> アキュテイン は通販での購
Wfgmvm
eriacta huddle - <a href="https://eriagra.com/asildigrah/">sildigra cluster</a> forzest keeper